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Abstract 


Workshops  in  the  late  1990’s  launched  the  commitment 
of  the  U.S.  Geological  Survey’s  Biological  Resources 
Division  (BRD)  to  develop  and  implement  decision 
support  systems  (DSS)  applications.  One  of  the  primary 
goals  of  this  framework  document  is  to  provide  suffi¬ 
cient  background  and  information  for  Department  of 
the  Interior  (DOI)  bureau  stakeholders  and  other  clients 
to  determine  the  potential  for  DSS  development.  Such 
an  understanding  can  assist  them  in  carrying  out 
effective  land  planning  and  management  practices.  This 
document  provides  a  definition  of  DSS  and  its  charac¬ 
teristics  and  capabilities.  It  proceeds  to  describe  issues 
related  to  meeting  resource  managers  needs,  such  as  the 
needs  for  specific  applications,  customer  requirements, 
information  and  technology  transfer,  user  support,  and 
institutionalization.  Using  the  decision  process  as  a 
means  to  guide  DSS  development  and  determine  users 
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needs  is  also  discussed.  We  conclude  with  information 
on  methods  to  evaluate  DSS  development  efforts  and 
recommended  procedures  for  verification  and  valida¬ 
tion. 

Key  words:  Adaptive  management,  Biological 
Resources  Division,  decision  making  process,  decision 
support  systems,  DSS,  resource  managers,  USGS. 


Introduction 


Since  the  time  that  people  began  living  in  ordered 
societies,  some  form  of  organized  decision  making  has 
been  an  integral  part  of  everyday  existence.  Before 
information  processing  technologies  were  developed, 
decisions  were  made  entirely  by  human  deliberation, 
aided  by  verbal  and  perhaps  some  printed  and  other 
visual  information.  In  today’s  environment  of  highly 
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developed  information  technologies,  many  powerful 
tools  exist  to  improve  and  refine  decision  making, 
although  it  remains  a  human-directed  process. 

In  the  1990’s,  significant  advances  in  computing 
power,  operating  systems,  memory  and  storage  capac¬ 
ity,  and  applications  software  made  it  feasible  to  use 
large  databases  and  geospatial  information  in  decision 
support  system  (DSS)  applications.  Combining  data, 
information,  and  computer-based  and  noncomputer- 
based  tools  and  services  within  a  structured  DSS 
framework  can  improve  both  the  process  and  outcomes 
of  decision  making.  In  the  current  world  of  connectiv¬ 
ity,  a  DSS  can  process  data  online  and  be  “seamlessly” 
accessed  by  many  different  hardware  and  software 
systems. 

Use  of  a  DSS  helps  resource  managers  better  define 
problems,  systematically  review  the  decisions  they 
make,  analyze  the  factors  that  influence  those  decisions, 
identify  information  that  is  available  with  respect  to 
these  factors,  and  predict  the  effects  of  making  deci¬ 
sions  with  and  without  desired  information.  A  DSS  can 
also  provide  a  framework  for  adaptive  management, 
information  feedback  loops,  and  continuous  improve¬ 
ment  of  the  decision  making  process. 

To  coordinate  ecological  DSS  efforts,  the  U.S. 
Geological  Survey’s  (USGS)  Office  of  Biological 
Informatics  and  Outreach  sponsored  a  Biological 
Resources  Division  (BRD)  Decision  Support  Systems 
Workshop  in  October  1998  (Getter  and  others,  1999).  In 
preparation  for  the  workshop,  a  questionnaire  was 
distributed  to  all  USGS  staff;  the  results  were  used  to 
construct  a  database  of  USGS  resources  related  to 
decision  support  systems.  A  Web  page  (http:// 
biology.usgs.gov/dss/meeting.html)  documents  the 
workshop  proceedings  and  the  results  of  the  USGS- 
wide  survey.  The  abstracts  included  in  the  proceedings 
can  give  insight  into  how  decision  support  systems  are 
currently  being  used. 

Workshops  in  the  late  1 990’s  launched  the  commit¬ 
ment  of  the  USGS  BRD  to  develop  and  implement  DSS 
applications  (see  Appendix  1  for  more  complete 
background  information).  The  BRD  assists  Department 
of  the  Interior  (DOI)  land  management  units  and 
resource  managers  with  a  wide  array  of  biological 
research  activities,  including  data  analysis,  modeling, 
and  decision  support  required  for  effective  land-use 
planning  and  management  of  parks,  refuges,  other 
resource  areas  and  their  environs  (USGS  Biological 
Resources  Division,  1996).  One  of  the  primary  goals  of 
this  framework  document  is  to  provide  sufficient 
background  and  information  for  DOI  bureau  stakehold¬ 
ers  and  other  clients  to  determine  the  potential  for  DSS 
development  that  can  assist  them  in  carrying  out 
effective  land  planning  and  management  practices.  The 


BRD  will  continue  to  play  a  significant  role  in  DSS 
research,  offering  the  biological  community  unique 
capabilities  for  problem  identification,  data  collection 
methodologies,  analysis  and  modeling,  evaluation  of 
solution  sets,  and  selection  of  optimal  solutions. 


Definition 


A  DSS  can  be  defined  generally  as  an  interactive, 
computer-based  tool  or  collection  of  tools  that  use(s) 
information  and  models  to  improve  the  process  and 
outcome  of  decision  making.  Many  definitions  of  DSS 
exist  (Bonczek  and  others,  1981;  Andriole,  1989;  Carter 
and  others,  1992;  Holsapple  and  Whinston,  1996).  Most 
identify  the  need  for  a  combination  of  database, 
interface,  and  model  components  directed  at  a  specific 
but  poorly  structured  problem.  Decision  support 
systems  can  be  simple  or  complex.  A  simple  decision 
support  tool  might  allow  the  user  to  view  and  query  the 
data,  such  as  in  a  database.  A  more  sophisticated  DSS 
incorporates  problem  structuring,  geospatial  data, 
models,  analytical  tools,  and  user  query  screens.  Such  a 
system  would  address  statistical  properties  of  results 
and  provide  reports,  output  maps,  and  graphs.  Geo¬ 
graphic  information  systems  (GIS)  provide  information 
in  a  spatial  context  that  can  help  managers  make 
decisions.  But  in  terms  of  these  definitions,  a  stand¬ 
alone  GIS  would  be  regarded  as  a  decision  support  tool , 
not  a  complete  decision  support  system ,  because  it  lacks 
both  support  for  the  use  of  problem  specific  models  and 
the  ability  to  assist  in  problem  definition. 

Before  categorizing  an  analysis  method  as  a  DSS  or  a 
decision  support  tool,  we  must  first  determine  whether 
the  user  is  being  provided  value-added  assistance  that 
transcends  the  original  intent  of  the  analytical  method. 
Databases,  or  some  forms  of  information  or  knowledge, 
are  the  building  blocks  for  a  DSS.  Developers  of  DSS 
can  add  valuable,  specific  biological  knowledge  to  these 
databases  as  well  as  provide  technological  expertise  for 
applying  the  data  and  for  constructing  models.  Making 
this  distinction — that  a  DSS  has  these  value-added 
qualities — helps  prospective  users  clarify  whether  they 
require  a  customized  DSS  to  address  their  problem  or 
whether  a  stand-alone  tool  will  be  sufficient. 

Some  DOI  bureau  needs  can  be  supported  simply  by 
providing  accurate  data  in  a  usable  format;  for  example, 
providing  landcover  data  for  the  area  of  interest  with  a 
simple  GIS  tool  to  view  and  overlay  the  data.  Land 
managers  can  examine  changes  over  time  and  compare 
this  information  with  other  relevant  data  to  help  them 
make  informed  decisions.  But  if  a  more  complicated 
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question  is  being  asked,  a  DSS  is  useful.  Decision 
Support  Systems  can  be  designed  to  query  the  user,  and 
through  models  and  analytical  tools,  integrate  that  user 
input  with  available  data.  The  DSS  returns  various 
scenarios  based  upon  the  value  of  parameters  entered 
by  the  user.  By  being  able  to  view  and  compare  the 
projected  results  of  the  various  simulations,  the  user  can 
make  a  more  informed  decision. 


Characteristics  and  capabilities 

In  addition  to  the  biological  sciences,  decision  support 
systems  that  integrate  GIS  and  simulation  models  can 
be  widely  applied  in  the  engineering,  planning,  and 
legislative  domains  (Brimicombe,  1992).  A  wide  variety 
of  databases,  models,  display  and  visualization  meth¬ 
ods,  and  other  tools  are  currently  available,  and  still 
more  are  under  development. 

Decision  support  systems  share  several  major 
characteristics: 

•  They  include  data,  images/graphics,  simulation 
models,  information,  or  knowledge. 

•  They  are  designed  to  assist  managers  in  the 
decision  process  for  semistructured  (or  unstruc¬ 
tured)  tasks  and  problems. 

•  They  support,  rather  than  replace,  managerial 
judgment. 

•  Their  objective  is  usually  to  improve  the  effective¬ 
ness  and  thoroughness  of  the  decisions  and 
sometimes  the  efficiency  with  which  the  decisions 
are  being  made. 

Typically,  decision  support  systems: 

•  Use  readily  accessible  and  affordable  hardware 
and  software. 

•  Provide  intuitive  user  interfaces  that  are  adaptable 
to  various  levels  of  sophistication. 

•  Provide  modularity  to  allow  incremental  develop¬ 
ment  and  building-block  interaction  with  other 
systems. 

•  Allow  for  Internet  connectivity  and  the  ability  to 
access  and  interact  with  remote  databases,  tools, 
models,  and  systems. 

•  Incorporate  interoperability  to  allow  use  of 
numerous  components  and  sources  of  data  tools, 
models,  and  systems. 

•  Presume  a  need  for  cooperative  development  with 
users. 

The  following  components  and  capabilities  may  be 
part  of  a  DSS: 

•  GIS  technology. 

•  Ability  to  accept  and  use  real-time  data. 

•  Ability  to  access  and  use  textual  materials. 


•  Ability  to  select  different  spatial  and  temporal 
scales. 

•  Modeling  and  simulation  tools. 

•  Mechanisms  to  allow  structured  problem  definition 
and  stakeholder  involvement. 

•  Visualization  tools  to  display  data,  relationships, 
and  projected  results. 

•  Tools  to  facilitate  adaptive  management  and 
monitoring. 

•  Means  to  depict  uncertainty  in  data,  relationships, 
or  results. 

•  Methods  to  treat  multiple  goals,  objectives,  and 
measures. 

•  Ability  to  create  and  store  scenarios. 

•  Ability  to  assemble  components  and  integrate  data 
as  needed. 

•  Ability  to  document  lineage  of  output. 

A  DSS  can  be  developed  as  a  network  system — one 
that  is  linked  to  large  data  warehouses  to  serve  multiple 
users — or  as  an  independent  system.  These  two  types  of 
DSS  are  adaptable  to  a  broad  range  of  uses.  In  both 
instances,  decision  makers  have  access  to  local  and 
external  data.  Network  decision  support  systems  can 
range  from  fairly  simple  systems  to  complex,  data- 
intensive,  and  analytically  sophisticated  information 
systems  that  provide  access  to  a  series  of  databases, 
models,  and  expert  systems.  Most  DSS  development  in 
the  BRD  to  date  has  focused  on  the  independent  DSS, 
primarily  because  consistent,  expansive,  seamless,  and 
interoperable  databases  and  framework  systems  are 
rare.  However,  framework  systems  with  interoperable 
databases  are  under  development  in  the  public  and 
private  sectors. 

Not  only  is  access  to  data  important,  but  the  quantity 
and  quality  of  available  data  and  the  degree  to  which 
data  are  georeferenced  are  important  aspects  of  a  DSS. 
Given  the  increasing  popularity  of  GIS,  more  geospatial 
data  are  becoming  available  and  usable  in  a  DSS; 
however,  while  data  are  abundant  for  some  natural 
resource  issues,  other  issues  appropriate  for  DSS 
application  have  minimal  data  available.  Insight  into  the 
quality  of  data,  partly  through  well-documented 
metadata,  will  help  in  evaluating  the  data’s  usefulness. 
Without  metadata,  data  accuracy  and  reliability  are 
unknown.  Indeed,  the  use  of  spatial  data  may  be 
inappropriate  when  addressing  a  specific  problem; 
instead,  other  reliable  information  and  knowledge  may 
be  required. 

It  is  easy  to  develop  unrealistic  expectations  when 
building  a  DSS.  Although  the  capabilities  of  a  DSS 
range  from  limited  to  highly  sophisticated,  even  the 
most  complex  DSS  will  not  be  designed  to  replace 
decision  makers,  nor  will  it  eliminate  poor  decisions  or 
make  good  decisions  without  the  input  of  experienced, 
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knowledgeable  people.  Managers  must  continue  to  ask 
the  correct  questions  and  draw  the  correct  conclusions 
from  the  information  they  receive.  If  lack  of  knowledge 
and  expertise  can  potentially  result  in  a  poor  decision,  a 
DSS  can  greatly  assist  the  user  by  offering  those 
missing  pieces  (Klein,  1998). 


Meeting  resource  managers  needs 


The  overriding  consideration  in  developing  an  effective 
DSS  is  to  involve  both  the  decision  makers  and  the 
research  and  development  scientists  in  the  entire 
process.  One  aspect  of  this  involvement  is  to  think  like 
an  application  builder  rather  than  a  programmer 
because  “application  builders  are  more  interested  in 
improving  decision  making  than  in  automating  it” 
(Carter  and  others,  1992).  Alter  (1978)  and  Finlay 
(1994)  identify  six  keys  to  success: 

1.  Devote  the  necessary  time  to  clearly  understand 
the  needs  of  users. 

2.  When  possible,  tailor  the  system  to  individual 
user  capabilities  and  management  style. 

3.  Maintain  close  contact  with  the  user  throughout 
development. 

4.  Provide  users  with  service  rather  than  a  product. 

5.  Design  a  simple  system  that  does  not  overload  the 
user. 

6.  Train  users  according  to  individual  levels  of 
expertise  and  rates  of  learning. 

In  addition,  we  recommend  development  of  a 
specific  team  (after  Sprague  and  Carlson,  1982;  Carter 
and  others,  1992)  that  should  include  upper-level 
management,  actual  users,  and  system  developers.  Such 
a  team’s  purpose  is  to  guide  the  project  from  inception 
through  implementation,  maintenance,  and 
reengineering,  focusing  especially  on  communication 
issues. 


The  need  for  specific  applications 

Our  national  wildlife  refuges,  national  parks,  and  other 
conservation  areas  are  under  increasing  development 
and  recreational  pressures,  resulting  in  a  need  for 
innovative  management  strategies.  The  DOI  will  need 
enhanced  abilities  to  provide  state-of-the-art  manage¬ 
ment,  protection,  and  interpretation  of  its  resources.  To 
meet  these  needs,  information  of  the  highest  quality 
must  be  collected  and  applied,  and  an  interdisciplinary, 
coordinated  approach  that  considers  both  socioeco¬ 
nomic  and  environmental  data  and  concerns  must  be 
used  in  the  decision-making  process. 


A  common  vocabulary  is  necessary  to  ensure  clear 
communication  in  discussions  of  DSS  development 
with  DOI  resource  managers  and  other  partners.  For 
example,  some  managers  may  interchangeably  use  the 
terms  “expert  system,”  “database,”  “GIS ,”  “information 
system,”  and  “DSS.”  Clearly  defining  at  the  planning 
stages  exactly  what  a  DSS  is — and  isn’t — will  ensure 
that  the  user’s  needs  and  expectations  are  addressed 
appropriately.  A  DSS  should  provide  a  more  powerful 
context  for  decision  making  (through  DSS  and  deci¬ 
sion-maker  interaction)  rather  than  be  an  end  in  and  of 
itself  (Power  and  Kaparthi,  1998). 

The  use  of  decision  support  systems  can  enable 
natural  resource  management  bureaus  to  make  decisions 
that  support  long-term  management  goals  and  priorities. 
Decision  analysis  and  support  systems  provide  struc¬ 
tures  for  systematically  analyzing  the  factors  involved 
in  decisions  and  determining  the  quantity  and  quality  of 
information  related  to  those  factors.  A  well  designed 
DSS  also  provides  decision  makers  with  frameworks 
that  assess  the  relative  risks  of  making  decisions  in  the 
absence  of  complete  information.  In  some  contexts,  the 
structure  ensures  that  the  decision-making  process  is 
institutionalized  and  archived. 

The  DOI  bureaus  would  benefit  by  having  decision 
support  systems  for  the  lands  and  resources  that  they 
manage.  For  example,  refuge  managers  and  national 
park  superintendents  often  need  to  compile  and  analyze 
information  on  natural  and  cultural  resources  within 
their  refuge  or  park  and  develop  “what  if’  scenarios  for 
management  alternatives.  Bureau  of  Reclamation 
managers  are  incorporating  a  DSS  in  reservoir  manage¬ 
ment.  The  Bureau  of  Land  Management  might  be 
interested  in  a  DSS  related  to  oil  development  and  sage 
grouse  ( Centrocercus  urophasianus )  habitat.  Decision 
support  tools  can  serve  as  electronic  ecosystem  ency¬ 
clopedias  that  can  be  used  to  integrate  and  visualize  a 
variety  of  data  such  as  vegetation  cover,  land-use 
practices,  roads,  stream  maps,  species  richness  maps  for 
plants  and  animals,  census  information,  and  output 
from  research  models.  Products  can  be  projected  or 
printed  in  formats  that  effectively  display  historical  and 
present  conditions  and  visualize  future  management 
goals  for  planners  and  the  public. 

Because  resource  managers  need  to  address  a  fairly 
broad  array  of  questions,  they  likely  want  a  desktop 
DSS  that  may  contain  a  variety  of  geographic,  scien¬ 
tific,  sociological,  and  economic  themes;  models; 
metadata;  text;  and  images.  The  most  functional 
systems  contain  procedures  for  mapping,  quantifying, 
graphically  displaying,  and  modeling  that  can  be 
understood  by  the  user  and  the  public.  Because  of 
varying  capabilities  and  needs,  creation  of  common 
platforms  and  interoperability  should  be  looked  at  as 
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goals,  not  absolute  requirements  in  the  development  of 
decision  support  systems  for  DOI  bureau  land  managers. 

To  ensure  a  client  driven  enterprise,  flexibility  is  of 
paramount  importance.  Some  clients  need  DSS  develop¬ 
ment  assistance  that  begins  at  the  most  basic  starting 
point.  Others  will  have  varying  degrees  of  DSS  compo¬ 
nent  development  already  under  way.  BRD  involvement 
will  depend  upon  each  client’s  unique  needs.  Nevertheless, 
DOI  clients  can  be  grouped  into  two  general  levels  of 
DSS  sophistication:  those  who  have  a  DSS  need  but  do 
not  have  any  existing  DSS  structure  in  their  organiza¬ 
tion  or  existing  tools,  and  those  who  have  some  kind  of 
operational  decision  structure  but  lack  integrated 
decision  support  tools.  A  primary  goal  of  USGS  support 
to  DOI  agencies  should  be  to  deliver  products  that 
reflect  state-of-the-art  biological  knowledge  in  a  mode 
the  client  can  use  most  effectively. 

A  DSS  based  on  spatial  data  could  provide  a  struc¬ 
ture  for  both  management  and  science  information 
(inventory,  monitoring,  and  research)  and  activities  or 
themes  for  the  decision  process.  Once  the  problem  has 
been  identified,  an  initial  step  may  be  to  compile  the 
required  geographic  themes  (e.g.,  digital  topographic 
data,  digital  orthophoto  quads  or  quarter  quads,  cover¬ 
ages  of  transportation,  hydrography,  vegetation, 
geology,  soils,  land  cover  and  land  use,  rectified  digital 
imagery  of  various  types,  boundaries,  management 
units,  points  or  areas  of  special  interest,  hazards,  etc.) 
for  DOI  lands.  Another  early  step  might  be  to  develop 
and  incorporate  the  tools  and  models  required  for  the 
analyses. 

The  BRD  inventory  contains  many  systems  based  on 
spatial  data  (Getter  and  others,  1999).  Some  are 
essentially  advanced  expert  systems  with  customized 
computer  programming  to  provide  an  interface  for 
resource  managers.  The  most  common  approach  at  the 
present  time  is  to  give  users  the  ability  to  download 
software  over  the  Internet  and  install  it  on  their  local 
personal  computers.  However,  full  functionality  of 
applications  is  often  not  available  over  the  Internet 
because  of  cost  and  programming  limitations,  process¬ 
ing  power  on  the  server,  and  speed  of  transmission.  A 
notable  exception  is  VegSpec,  an  Internet-based  expert 
system  (ironwood.itc.nrcs.usda.gov/scripts/ndisapi.dll/ 
vegspec21/pagVegspecStart)  developed  cooperatively 
by  the  USGS,  U.S.  Army  Corps  of  Engineers,  and  the 
U.S.  Department  of  Agriculture  Natural  Resources 
Conservation  Service.  This  application  assists  the  land 
manger  in  finding  and  selecting  adapted  plants  for  use 
in  addressing  vegetation  restoration  problems.  Although 
largely  a  knowledge-based  system,  VegSpec  does  have 
data-based  components,  as  it  accounts  for  soil  types  and 
topographic  variables  (elevation,  slope,  and  aspect). 


Customer  requirements 

Ultimately,  the  tools  and  capabilities  developed  by  the 
USGS  must  be  of  practical  use  to  DOI  managers  and 
other  partners.  However  complex  or  simple  a  DSS  may 
be,  it  must  address  the  needs  of  land  and  resource 
managers. 

Decisions  made  by  natural  resource  managers  are 
inherently  complex.  Thousands  of  natural  features  and 
processes — attributes  of  a  natural  system  that  vary 
across  time  and  space — might  influence  any  given 
decision.  Yet  decisions  must  be  made  every  day,  and 
these  decisions  must  be  defensible.  A  natural  resource 
manager  requires  a  clear  understanding  of  the  current 
and  future  ecological  ramifications  of  all  decisions. 

If  a  DSS  is  to  be  a  trusted  source  of  information, 
some  basic  issues  must  first  be  addressed  before 
development  and  implementation.  Although  technologi¬ 
cal  change  throughout  society  has  been  monumental  in 
recent  years,  natural  resource  managers  in  DOI  have  not 
necessarily  been  leaders  in  using  advanced  technologies 
(Biddle  and  others,  1995).  Here  we  provide  consider¬ 
ations  to  determine  whether  the  time  and  resources 
required  will  meet  DOI  needs  in  a  cost-effective 
manner. 


Information  and  technology  transfer 

Effective  and  maximal  use  of  decision  support  tools 
involves  sharing  information  about  the  existing  tools. 
Managers  must  be  made  aware  of  the  existence  of  such 
tools,  including  their  strengths  and/or  limitations  and 
how  they  can  best  be  used  to  solve  problems. 

Managers  must  be  informed  of  what  to  expect  from  a 
specific  DSS.  Publication  of  individual  applications  on 
the  Web  and  elsewhere  will  provide  a  method  of 
continual  communication  to  help  maximize  the  useful¬ 
ness  of  decision  support  systems.  Open  communication 
should  occur  at  all  levels — across  agencies  at  the 
manager/scientist/technologist  level,  as  well  as  up  and 
down  the  chain  of  command — so  that  all  participants  in 
the  decision-making  process  are  kept  adequately 
informed  of  what  information  is  available  and  how  it 
can  be  used. 

Many  decision  support  systems  incorporate  the  most 
current  and  sophisticated  technologies  available.  While 
use  of  these  technologies  enhances  the  capabilities  of 
these  systems,  more  specialized  hardware  and  knowl¬ 
edge  may  be  required  to  use  such  a  DSS.  Before 
developing  a  DSS,  we  must  be  sure  that  the  intended 
users  are  ready  for  the  specific  technology  by  assessing 
the  hardware  and  software  capabilities  of  the  intended 
users  and  by  determining  the  type  of  scientific  expertise 
the  DSS  requires.  Providing  a  tool  that  has  the  desired 
capabilities  but  is  not  usable  by  the  manager  is,  at  best, 
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a  wasted  effort.  At  worst,  it  can  discourage  managers 
from  attempting  to  use  these  kinds  of  tools  even  when 
they  eventually  acquire  the  needed  capabilities.  The 
DSS  user  should  not  be  dependent  upon  the  system 
developer  to  successfully  use  the  system. 

Despite  the  efforts  outlined  above,  there  will  likely 
still  be  resistance  on  the  part  of  some  potential  users 
who  are  hesitant  to  take  full  advantage  of  available 
decision  support  tools  and  systems.  The  user  without 
technical  expertise  may  be  reluctant  to  use  a  tool  not  yet 
fully  understood.  Managers  and  other  decision  makers 
will  have  to  be  convinced  that  the  tool  does  an  adequate 
job  accounting  for  the  meaningful  assumptions  and 
addressing  the  prominent  features  of  a  problem. 
Furthermore,  a  basic  tenet  of  decision  support  systems 
is  that  their  focus  is  on  support,  i.e.,  managers  are  still 
responsible  for  making  the  actual  decisions.  Such 
software  systems  are  not  intended  to  supplant  managers, 
but  rather  to  assist  them. 


User  support 

Once  the  user  adopts  a  DSS  for  a  specific  application,  it 
is  crucial  that  the  BRD  continue  to  provide  user  support 
and  play  a  role  in  the  DSS  implementation  stage.  The 
majority  of  decision  support  systems  will  require  some 
degree  of  training  and  technical  support  to  bring  users 
to  a  level  where  they  are  able  to  operate  them  effec¬ 
tively  and  take  full  advantage  of  all  their  features.  Time 
allocated  to  provide  training  and  technical  assistance 
can  be  a  sizeable  commitment,  but  we  must  be  prepared 
to  fulfill  this  needed  role.  Once  the  managers  and  their 
staff  have  become  comfortable  using  the  tool,  training 
and  technical  assistance  will  subside.  There  may  be  a 
demand  for  “on  call”  duty  to  help  with  particular 
problems,  but  the  need  for  assistance  should  be  minimal 
past  the  initial  stages  of  training  and  technical  assis¬ 
tance. 


Institutionalization 

The  previous  two  sections  point  out  some  of  the  issues 
that  will  need  to  be  addressed  by  both  the  client  (e.g., 
DOI  land  management  agencies)  and  DSS  developer 
organizations  (e.g.,  BRD  or  USGS).  For  DSS  develop¬ 
ment  and  use  to  successfully  advance  at  reasonable  and 
sustainable  rates,  organizations  must  commit  to  institu¬ 
tionalizing  this  activity.  The  current  situation  with  DSS 
is  very  similar  to  that  of  GIS  in  the  early  to  mid  1980’s. 
There  were  “pockets  of  GIS  activity”  in  many  organiza¬ 
tions  but  little  institutional  support  to  foster  its  develop¬ 
ment  and  use  or  “glue”  to  make  the  disparate  GIS 
activities  into  a  coordinated,  supported,  valued,  and 


viable  organizational  enterprise.  Once  GIS  was  institu¬ 
tionalized  within  the  various  organizations,  it  flour¬ 
ished.  Other  factors  (e.g.,  less  costly  and  more  powerful 
hardware  and  software)  also  contributed  to  the  success¬ 
ful  adoption  and  use  of  GIS  during  that  rapidly  evolving 
period  for  GIS,  but  institutionalization  was  the  catalyst 
needed  to  make  GIS  “standard  operating  procedure” 
within  public  and  private  organizations.  The  same 
situation  pertains  to  DSS. 

A  full  elaboration  on  what  is  needed  to  institutional¬ 
ize  DSS  within  BRD  or  USGS  and  client  organizations 
will  vary  among  organizations  and  is  not  appropriate 
here.  However,  some  of  the  more  generally  applicable 
institutionalization  issues  that  will  need  to  be  addressed 
are  budget,  staff,  organizational  buy-in  and  oversight, 
information  and  technology  transfer,  user  support,  and 
coordination.  As  such,  BRD  has  started  the  institution¬ 
alization  process,  but  there  is  still  much  more  that  needs 
to  be  done. 

The  decision  process  as  a  component 
of  decision  support  system 
development 

The  process  of  decision  making  is  generally  recognized 
to  include  four  stages  (Carter  and  others,  1992).  First  is 
intelligence  gathering.  This  gathering  includes  an 
analysis  of  the  decisions  to  be  made,  how  they  will  be 
made,  and  what  information  pertinent  to  those  decisions 
is  needed  and  where  it  might  be  available.  An  important 
component  of  this  first  stage  is  getting  information  into 
a  format  that  is  usable,  accessible,  and  archived.  Second 
is  designing  alternatives .  This  includes  generating 
possibilities  along  with  analyzing  their  potential 
outcomes.  Often,  the  uncertainty  associated  with 
underlying  information  and  the  propagation  of  risks 
associated  with  decisions  will  be  analyzed  in  this  stage. 
Third  is  choosing  among  alternatives  and  is  the  process 
of  actually  making  a  decision.  Fourth  is  the  on-the- 
ground  implementation  of  the  decision  along  with  an 
evaluation  of  that  decision  in  terms  of  achieving  goals. 
Simon’s  (1960)  early  seminal  work  on  decision  making 
did  not  explicitly  recognize  evaluation  as  a  stage  and 
combined  on-the-ground  implementation  with  stage 
three. 

Recent  work  by  Cleaves  (1999)  in  the  realm  of 
collaborative  decision  making  has  produced  two 
additional  steps  that  precede  Carter’s  first  step  of 
intelligence  gathering.  These  steps  are  process  mapping 
(i.e.,  deciding  how  the  decision  is  to  be  made  and  who 
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will  make  it)  and  problem  framing  (i.e.,  describing  the 
problem  to  be  solved  or  opportunity  to  be  captured  in 
terms  of  yardsticks  for  success,  reference  points  from 
which  to  mark  improvements,  and  boundaries  that 
describe  how  much  of  the  situation  is  “fair  game”  for  a 
generation  of  alternatives).  Process  mapping  and 
problem  framing  are  extremely  important  first  steps 
when  working  in  a  collaborative  decision-making 
environment  for  place-based  issues  that  require 
multiagency  management  programs,  like  the  Mojave 
Desert  Ecosystem  Program  or  that  require  multiscale 
(e.g.,  local,  county,  state,  and  regional)  planning 
activities,  like  the  South  Florida  Ecosystem  Program  or 
the  Pacific  Northwest  ecosystem. 

Although  DSS  development  must  address  all  relevant 
stages,  computerized  tools  cannot  necessarily  make 
equally  important  contributions  to  each  stage.  For 
example,  an  analysis  of  decisions  to  be  made,  which 
will  continue  to  be  the  ultimate  foundation  of  DSS 
development,  will  likely  remain  the  purview  of  research 
specialists,  management  and  systems  analysts,  and 
human  dimensions  scientists  (although  sometimes  the 
analysis  of  decisions  can  be  part  of  the  DSS).  On  the 
other  hand,  more  technologies  are  being  developed  that 
are  proving  useful  in  identifying  problems,  setting 
objectives,  and  facilitating  group  decisions.  Information 
technologies,  especially  when  coupled  with  the  Internet, 
can  make  a  successful  contribution  to  intelligence 
gathering.  Knowledge  engineering,  metadata  develop¬ 
ment,  and  analysis  of  remotely  sensed  data  fit  this  stage 
of  the  decision-making  process. 

When  designing  alternatives,  the  utility  of  advanced 
methods  and  technologies  is  often  most  apparent. 
Examples  include  statistical  analysis,  rule-based 
modeling,  GIS,  and  Bayesian  belief  networks.  The  role 
of  a  DSS  in  evaluating  decisions  is  often  overlooked, 
although  it  clearly  can  make  significant  contributions. 
This  is  especially  true  in  an  adaptive  management 
framework,  where  quantifying  the  value  of  new 
information  is  critical.  A  key  concept  in  adaptive 
management  is  that  new  information  affects  the 
uncertainties  associated  with  future  alternatives. 


Framing  the  decision  processes  of  natural  resource 
managers 

Decision  support  systems  can  be  useful  for  a  variety  of 
reasons.  Klein  (1998)  identified  three  causes  for  “why 
good  people  make  poor  decisions”:  (1)  they  lack 
technical  experience,  (2)  they  lack  information,  and  (3) 
they  use  inadequate  mental  simulation  of  possible 
scenarios.  In  addition  to  helping  to  remedy  these  three 
difficulties,  a  DSS  can  also  assist  with  high  volume  or 


complicated  computations  (e.g.,  population  viability 
analysis  and  hydrologic  modeling). 

The  uncertainty  associated  with  decision  making, 
particularly  with  evaluating  data,  is  another  problem 
that  can  be  addressed  by  a  DSS.  For  example,  when 
land  managers  must  make  choices  about  habitat 
manipulations,  there  invariably  is  uncertainty  either 
about  vegetation  and  wildlife  parameters  or  about  the 
ecological  relationships  affected  by  such  manipulations. 
The  effects  of  prescribed  fire  in  native  prairie  on 
mallard  (Anas  platyrhynchos)  nesting  success  may  be 
well  known  in  Stutsman  County,  North  Dakota,  but  a 
biologist  in  the  Rainwater  Basin  of  Nebraska  may  be 
trying  to  decide  whether  to  bum  native  grasses  for  blue¬ 
winged  teal  (Anas  discors)  nesting.  When  the  uncer¬ 
tainty  is  primarily  about  the  quality  of  information 
entered  into  the  system,  DSS  should  generate  potential 
alternatives  and  recommend  how  to  evaluate  them. 
When  the  uncertainty  is  associated  with  the  outcome  of 
potential  management  actions,  a  system  must  generate 
and  evaluate  management  options  (Adelman,  1992). 
Finally,  the  issue  being  addressed  by  a  manager  can  be 
so  complex  that  no  one  individual  can  have  a  compre¬ 
hensive  view  of,  or  even  conceptualize,  the  whole 
problem  or  venue  (Brehmer,  1991;  Boland  and  others, 
1992).  For  example,  how  does  the  need  for  winter  food 
for  lesser  snow  geese  (Chen  caerulescens)  in  the 
Central  Valley  of  California  relate  to  (1)  quality  and 
quantity  of  fall  habitat  at  migration  stopover  sites  in  the 
Klamath  Basin  of  Oregon;  (2)  gosling  production  at 
Wrangle  Island,  Alaska,  or  at  Banks  Island,  Northwest 
Territories,  during  the  previous  summer;  (3)  early 
winter  weather  patterns  on  the  Fraser  River  Delta, 
British  Columbia;  and  (4)  prevalence  of  avian  disease  in 
southern  California?  Ecological  processes  are  inher¬ 
ently  complex,  and,  therefore,  so  are  many  of  the 
decisions  resource  managers  must  make. 

Decision  making  in  the  everyday  world  tends  to  be 
influenced  by  at  least  eight  factors  (Orasanu  and 
Connolly,  1993):  (1)  poorly  defined  problems;  (2) 
uncertain,  changing  environments;  (3)  shifting,  ill- 
defined,  or  competing  goals;  (4)  immediate  actions  that 
must  occur  before  a  decision  can  be  formed;  (5)  high- 
stress  situations;  (6)  high  stakes;  (7)  multiple  people; 
and  (8)  organizational  expectations.  Consider,  for 
example,  how  many  or  if  all  of  these  factors  are  in 
effect  as  a  fisheries  manager  assembles  the  staff  to 
discuss  the  potential  benefits  that  will  accrue  from 
planting  willows  under  differing  spring  weather 
scenarios  to  help  stabilize  the  degraded  banks  of  a 
creek.  Or,  consider  how  these  eight  factors  describe  the 
decision  making  environment  that  surrounds  planning 
how  much  refuge  staff  time  and  budget  will  be  neces¬ 
sary  for  inventorying  piping  plover  (Charadrius 
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melodus)  reproduction  this  year.  Almost  every  decision 
natural  resource  managers  must  consider  is  influenced 
by  these  factors.  Only  infrequently  are  natural  resource 
managers  faced  with  the  isolated  and  incremental 
decisions  that  might  typify  some  aspects  of  a  manufac¬ 
turing  assembly  line. 

Determining  decision  support  systems  needs 

Cleaves  (1999)  has  delineated  four  questions  that 
provide  a  valuable  basis  from  which  to  design  decision 
support  systems: 

•  What  decisions  need  support?  Usually,  a  DSS, 
almost  by  definition,  addresses  poorly  structured 
problems  (Bonczek  and  others,  1981;  Sprague  and 
Carlson,  1982;  Mallach,  1994;  Holsapple  and 
Whinston,  1996)  or  at  least  very  complex  prob¬ 
lems  (Carter  and  others,  1992).  A  different 
approach  (Andriole,  1989)  recommends  applying  a 
DSS  to  better  defined  and  less  complex  problems 
to  increase  the  probability  of  success.  In  any  case, 
the  decision  to  be  supported  must  be  clearly 
delineated. 

•  What  parts  of  the  decision-making  process  will  be 
supported  by  the  system? 

•  What  kind  of  support  is  needed? 

•  How  will  the  DSS  fit  into  the  decision-making 
process? 

Useful  DSS  must  result  in  one  or  more  of  the 
following  outcomes.  First,  more  effective  decisions  can 
be  made  that  allow  goals  to  be  met,  such  as  attaining 
population  objectives,  improving  habitat  conditions,  or 
increasing  the  quality  of  monitoring  data.  Second,  more 
efficient  decisions  can  result,  such  as  decisions  requir¬ 
ing  less  time,  fewer  or  lower  ranking  personnel,  and  the 
input  of  less  data  and  information.  Third,  risk  associ¬ 
ated  with  future  options  can  be  estimated  and  thus 
might  be  decreased.  Typically,  such  risk  is  lowered  by 
decreasing  the  uncertainty  associated  with  data, 
information,  knowledge,  and  analyses  upon  which 
decisions  are  based.  Low  risk  alternatives  would 
minimize  the  chances  of  future  catastrophes.  Fourth,  an 
increase  in  the  ability  to  understand  complex  decision 
frameworks  can  result,  along  with  an  accompanying 
ability  to  explain  management  decisions  to  stakehold¬ 
ers. 

Carter  and  others  (1992)  provide  a  slightly  different 
perspective  on  DSS.  They  state  that  good  systems 
minimize  the  probabilities  of  four  things:  (1)  making  a 
poor  decision,  (2)  missing  a  good  alternative,  (3) 
making  a  decision  at  a  bad  time,  or  (4)  focusing  on  the 
wrong  problem. 


Launching  the  research  and  development  process 

Numerous  variations  exist  for  developing  a  DSS  (see 
table  1).  Based  on  a  review  of  this  information,  we 
recommend  the  following  DSS  research  and  develop¬ 
ment  process,  which  recognizes  the  various  stages  of 
the  decision-making  process  described  previously  (see 
Decision  Process  as  a  Component  of  DSS  Development 
section).  It  also  recognizes  a  distinction  between 
decision  making  itself  and  development  of  DSS  to 
support  those  decisions. 

1.  a.  Evaluate  the  existing  decision-making  process 
used  by  managers  that  might  benefit  from 
decision  support  (i.e.,  intelligence  gathering). 
Formulate  one  or  more  testable  hypotheses  about 
where  and  how  decision  support  would  be  useful 
in  particular  venues,  recognizing  that  a  DSS 
should  address  poorly  defined  problems.  Ex¬ 
ample:  Identify  formal  policies,  individual 
values,  available  information,  computerized 
tools,  professional  networks,  and  other  consider¬ 
ations  that  might  assist  or  constrain  a  forest 
supervisor’s  decision  to  close  or  open  logging 
roads  for  motorized  recreation, 
b.  Conduct  research  that  will  guide  development  of 
specific  decision  support  tools  by  defining  the 
decision-making  process  of  management  users. 
Example:  A  park  biologist  might  want  to  assess 
potential  effects  of  campground  development  on 
riparian  sites.  Identify  the  process  used  by  the 
biologist  to  determine  the  value  of  riparian 
habitat  values  and  how  that  process  could  be 
improved  with  decision  support  tools. 

2.  Use  state-of-the-art  technologies,  operations 
research,  and  management  methods  for  building 
and  delivering  decision  support  systems.  Ex¬ 
ample:  Constraint  satisfaction  programming 
methods  (see  glossary)  have  been  used  to  provide 
guidance  for  managers  on  how  to  manage 
seasonal  wetlands  for  certain  moist  soil  charac¬ 
teristics  (see  http://www.mesc.usgs.gov/msma/). 

3.  Implement  decision  support  tools,  iteratively 
conducting  validation  and  verification.  Example: 
During  development  of  a  DSS  for  trumpeter 
swan  ( Cygnus  buccinator )  management,  early 
versions  were  made  accessible  via  the  World 
Wide  Web  to  managers  and  biologists  to  gain 
their  input  about  the  reliability  of  contributed 
data  (see  http://swan.msu.montana.edu/cygnet). 

4.  Evaluate  decision  support  tools  by  testing 
previously  developed  hypotheses.  Was  the 
research  able  to  accurately  predict  where  and 
how  such  tools  would  be  helpful  to  managers  in 
making  decisions?  Did  more  effective  or  efficient 
decisions  result?  Example:  Maybe  we  hypothesized 
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Table  1.  Various  approaches  to  DSS  development  and  application. 

Andriole  (1989) 

Carter  et  al.  (1992) 

Finlay  (1994)' 

Sage  (1991) 

Sprague  and 
Carlson  (1982) 

Stuth  and 
Smith  (1993)2 

Requirements 

analysis 

System  definition 

Feasibility 

Requirements 

specification 

Subproblem 

identification 

Broad 

requirements 

specification 

Modeling  (of  the 
system  itself) 

Iterative 

development 

Analysis 

Preliminary 

conceptual 

design 

Small  but 
usable  system 
development 

Prototype 

development 

Methods 

selection 

Continual 
implementation 
and  maintenance 

Design 

Detailed  design, 
testing,  and 
implementation 

Cyclical 
refinement, 
expansion,  and 
modification  of 
the  system 

Prototype 

testing 

Software 

selection/design 

Programming 

Operational  test 
and  evaluation 

Constant 

evaluation 

Detailed 

specification 

Hardware 

selection/design 

Testing 

Operational 

deployment 

Pilot  system 
design  and 
implementation 

System  packaging 

Documentation 

Field  trials 

System  transfer 

Implementation 

Evaluation  and 
iteration 

Evaluation 

Review/  evaluation 

Operational 
systems  design 
and 

implementation 

Feedback 

Maintenance 

Operational 
use  and 
feedback 

User  support, 
review,  and 
iteration 

Evolutionary 

development 

Widening  use 


1  Each  of  these  steps  is  further  subdivided  into  6-21  substeps. 

2  The  authors  credit  Eason  (1988);  their  process  is  not  necessarily  linear. 


that  habitat  acquisition  biologists  needed  an 
expert  system  and  GIS  combination  to  assess  in 
what  specific  areas  the  greatest  benefit  would 
accrue  from  their  efforts  to  acquire  wetland 
habitat.  Did  the  system  that  was  provided  lead  to 
recommendations  that,  when  implemented, 
resulted  in  a  stabilization  of  waterfowl,  rail 
(Family  Rallidae),  and  shorebird  numbers  in  a 
county  where  they  had  earlier  been  decreasing? 

5.  Institutionalization  of  the  DSS  enterprise 

throughout  the  organization  (e.g.,  budget,  staff, 


organizational  buy  in  and  oversight,  coordination, 
information  and  technology  transfer,  support,  etc.). 

Developing  software:  evolutionary  prototyping 

We  recommend  the  use  of  evolutionary  prototyping  for 
DSS  development  (Sprague  and  Carlson,  1982;  Carter 
and  others,  1992).  This  is  a  step-by-step  process  that 
integrates  planning,  developing,  validating,  delivering, 
and  supporting  software  systems  and  is  based  on 
extensively  involving  users  throughout.  It  can  be 


The  decision  process 


9 


thought  of  as  focusing  on  the  development  of  an  early 
prototype,  even  during  the  planning  stages,  and  then 
continually  refining  it  to  its  eventual  implementation. 
Our  overall  recommended  process  for  DSS  develop¬ 
ment  is  summarized  in  figure  1. 

Evaluating  DSS  development  efforts  - 
the  need  for  verification  and 
validation 

Why  should  one  evaluate  decision  support  systems 
and  expert  systems ?  The  answer  is,  quite  simply,  to 
increase  the  probability  that  they  will  be  used  and 
effective  (Adelman,  Evaluating  Decision  Support 
and  Expert  Systems,  1992). 

The  scientific  literature  is  particularly  devoid  of  work 
on  the  evaluation  of  decision  support  systems.  This  can 
be  traced,  at  least  partially,  to  problems  associated  with 
verification  and  validation  of  such  complex  software 
systems.  On  the  other  hand,  much  can  be  learned  by 
examining  the  analogous,  and  fairly  rich  literature  on 
verification  and  validation  of  expert  systems  and  other 
artificial  intelligence  methods,  which  are  quite  often 
types  of  decision  support  systems  (see  Cohen  and 
Howe,  1989;  Bahill,  1991;  Grogono  and  others,  1991; 
Gupta,  1991;  Hamilton  and  others,  1991;  Adelman, 
1992;  O’Leary,  1994). 

Verification  and  validation 

Wallace  and  Fujii  (1989)  define  verification  and 
validation  of  software  as  analysis  and  testing  “to 
determine  that  [the  software]  performs  its  intended 
functions  correctly,  to  ensure  that  it  performs  no 
unintended  functions,  and  to  measure  its  quality  and 
reliability.”  Verification  and  validation  are  often 
considered  together  under  the  heading  of  evaluation. 
Evaluation  should  be  part  of  the  development  process, 
and  evaluators  should  specifically  be  part  of  the 
development  team  (Adelman,  1992). 

“Verification”  means  ensuring  that  the  system  is 
internally  complete,  coherent,  and  logical,  essentially 
from  a  modeling  and  programming  perspective. 
“Validation”  means  examining  whether  the  system  is 
accurate,  realistic,  and  useful  to  the  user  or  decision 
maker  (Geissman  and  Schultz,  1988).  O’Keefe  and 
others  (1987)  state:  “Validation  means  building  the  right 
system.  Verification  means  building  the  system  right  ” 
For  example,  national  park  biologists  might  be 
interested  in  a  system  that  helps  them  design  water 


quality  monitoring  schemes  for  spring-fed  streams  to 
determine  appropriate  road  improvement  procedures. 
Verification  might  involve  ensuring  that  the  system  will 
access  the  correct  data  and  that  the  graphical  results  will 
be  of  the  same  statistical  precision  and  accuracy  as 
represented  in  the  textual  presentation.  Validation  might 
check  that  the  system  is  reporting  those  figures  in  a 
fashion  usable  by  the  biologist,  likely  requiring  expla¬ 
nation  of  the  tests  used  and  why  they  are  appropriate. 
More  importantly,  validation  would  ensure  that  the 
correct  monitoring  parameters  are  used  to  help  the  park 
make  the  determinations  that  they  need  regarding  road 
improvements. 


An  overview  of  methods 

Following  Adelman  (1992),  successful  implementation 
of  decision  support  and  expert  systems  hinges  on 
incorporating  three  evaluation  procedures:  (1)  those  that 
examine  the  logical  consistency  of  the  system  algo¬ 
rithms  themselves  (verification),  (2)  those  that  empiri¬ 
cally  test  the  predictive  (or  ecological)  accuracy  and 
effectiveness  of  the  system  (validation),  and  (3)  those 
that  document  user  satisfaction  and  meet  user  needs. 

Stuth  and  Smith  (1993)  followed  the  ideas  of  Eason 
(1988)  and  recommend  iterative  prototyping  methods 
for  DSS  development.  When  using  such  methods, 
verification  and  validation  are  part  of  the  iterative 
process  of  system  development.  Verification  should  be 
performed  at  the  stage  prior  to  delivery  of  a  working 
system  to  users,  even  if  only  a  prototype  system. 

General  validation  might  be  done  at  this  stage  as  well, 
with  more  detailed  efforts  performed  once  an  operating 
system  is  delivered.  If  one  subscribes  to  the  concept  that 
software  development  can  be  a  dynamic  process,  then 
verification  and  validation  are  vital  parts  of  that 
continued  process  which  seeks  regular  refinements 
(Carter  and  others,  1992;  Stuth  and  Smith,  1993). 

Sprague  and  Carlson  (1982)  recommend  that 
organizations  building  their  first  DSS  recognize  that  it  is 
essentially  a  research  activity  and  that  evaluation  should 
center  on  a  general  value  analysis  as  defined  by  Keen 
(1981).  This  consists  of  four  steps:  (1)  identify  the 
potential  benefits,  (2)  determine  the  maximum  cost 
allowable  for  development,  (3)  develop  a  prototype,  and 
(4)  determine  the  benefits  in  relation  to  costs.  They  state 
that  iterative  prototyping  will  ensure  a  quality  product 
from  the  managers’  perspectives  but  recognize  the 
qualitative  nature  of  such  evaluation.  It  is  imperative 
that  analytic  and  quantitative  rigor  be  added  beyond  the 
“soft  testimonials”  often  seen  (Andriole,  1989;  Cohen 
and  Howe,  1989).  Sensitivity  analysis  can  be  a  powerful 
tool  for  validation,  especially  for  heuristic-based 
systems  and  for  systems  where  few  or  no  test  cases  are 
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Study  the 
decision 


Identify  an  important  subproblem  on 


process  whjch  to  focus 


Conclude  with  hypothesis  about  where 
a  DSS  would  help 


Determine  feasibility  of 
developing  a  DSS 


Flowchart  the 
overall  system 


Develop  a 
prototype 


Refine,  modify, 
and  expand  the 
system 


Validation 


Conduct  additional 
systems  requirements 
analysis 


Evolutionary  Prototyping 


Extensive  user 
involvement 
throughout  this 
cycle 


Figure  1 .  A  recommended  process  for  decision  support  system  development. 
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available  for  comparison  (O’Keefe  and  others,  1987; 
Bahill,  1991).  Another  issue  suggested  by  Rushby 
(1991)  is  that  it  is  necessary  to  show  not  only  how  well 
a  system  performs  but  also  to  show  that  it  can  avoid 
making  a  catastrophic  recommendation.  This  feature  is 
important  in  many  natural  resource  venues  because  of 
the  great  concern  for  long-term,  irretrievable  ecological 
changes. 

Consider,  for  example,  the  sensitive  issues  associated 
with  determining  the  impact  of  dredging  on  mussels  in 
a  watershed.  Obviously,  a  DSS  trying  to  support  such 
decisions  needs  to  maximize  the  opportunities  for 
maintaining  healthy  mussel  populations.  However, 
doing  so  might  increase  the  probability  of  a  catastrophic 
response  in  the  mussel  population  should  some  dynamic 
environmental  variable  rapidly  change.  Is  the  DSS 
capable  of  handling  the  scenario  of  a  500-year-fre¬ 
quency  flood  event’s  impact  on  mussels?  For  a  species 
whose  population  is  endangered,  such  as  the  dwarf 
wedge  mussel  ( Alasmidonta  heterodon),  this  can  be  a 
critical  question. 

For  many  reasons,  verification  and  validation  of 
knowledge-based  and  other  decision  support  systems 
are  more  problematic  than  for  other  modeling  efforts 
(Gupta,  1991).  Under  some  conditions,  modeling 
research  can  test  performance  against  a  preselected 
standard.  With  natural  resource  issues,  such  a  standard 
often  does  not  exist.  This  is  particularly  true  with  near 
real-time  decision  support  that  is  expected  to  predict 
and  guide  future  scenarios  while  those  scenarios  are,  in 
fact,  unfolding.  Also,  not  only  should  a  system  handle 
the  most  common  cases,  but  it  ought  to  also  be  able  to 
model  extreme  events  as  well.  Extreme  events  are  not 
only  common  in,  but  often  have  profound  effects  on 
ecological  systems.  For  example,  many  habitats 
throughout  North  America  maintain  their  diagnostic 
characteristics  through  a  dependence  upon  fire.  In  cases 
such  as  tall  grass  prairie,  fires  may  be  thought  of  as  a 
common  occurrence,  and  this  type  of  event  can  be 
replicated  in  a  DSS  somewhat  straightforwardly.  But 
what  happens  in  other  places,  such  as  the  Northern 
Rockies,  where  the  time  elapsed  between  fires  may  be 
many  decades  or  longer?  In  addition  to  being  equipped 
to  address  “normal”  conditions,  a  DSS  for  such  situa¬ 
tions  must  be  versatile  enough  to  simulate  various  fire 
regimes  and  the  probabilities  associated  with  ideal 
conditions  and  all  other  possibilities,  and  flexible 
enough  to  handle  sudden  corrections  when  an  unex¬ 
pected  drought  or  wet  period  alters  fire  potentials 
beyond  what  was  anticipated.  Designing  formal 
evaluations  to  ensure  DSS  are  powerful  and  dynamic  is 
difficult. 

It  is  sometimes  possible  to  test  expert  system 
performance  against  an  independent  panel  of  experts 


(O’Keefe  and  others,  1987).  Two  concerns  must  be 
addressed,  however.  First,  the  panel  of  experts  needed 
for  such  an  evaluation  must  not  be  the  same  people  who 
were  connected  to  system  development  itself,  or 
confounding  effects  could  be  introduced  that  would 
prevent  reasonable  experimental  design.  Second,  one  of 
the  basic  tenets  of  using  a  DSS  for  complex  issues  is 
that  such  questions  are  beyond  the  capability  of 
individuals  to  conceptualize  and  solve  (Brehmer,  1991; 
Boland  and  others,  1992). 

Wallace  and  Fujii  (1989)  provide  a  comprehensive 
matrix  of  41  techniques  and  tools  that  can  be  applied  to 
10  verification  and  validation  issues.  Cohen  and  Howe 
(1989)  take  a  slightly  different  approach  but  also 
discuss  evaluation  from  the  perspective  of  the  software 
development  life  cycle.  Their  emphasis  is  on  empirical 
studies  to  accomplish  such  evaluation,  whether  one  is 
focusing  on  verification  or  validation.  Specifically  for 
knowledge-based  systems,  Murrell  and  Plant  (1997) 
provide  a  categorization  of  145  different  automated 
techniques  for  testing  such  systems. 


Recommended  procedures 

The  following  eight  general  steps  should  be  used  for 
verification  (Geissman  and  Schultz,  1988;  Cohen  and 
Howe,  1989;  Wallace  and  Fujii,  1989;  Grogono  and 
others,  1991;  Sage,  1991 ;  Adelman,  1992;  Finlay, 

1994).  Whenever  possible,  experimental  testing  of  the 
complete  system,  or  components  thereof,  should  be 
performed. 

1 .  Examine  the  high-level  (overall  system)  design. 

•  Are  all  requirements  covered,  and  are  they  broken 
down  into  reasonable  components? 

•  Are  any  components  redundant? 

•  Do  individual  components  request  appropriate 
inputs  and  deliver  correct  outputs? 

•  Are  uncertainties  propagated  correctly  among 
components? 

2.  Examine  each  component  to  ensure  that  it  carries  out 
what  is  requested  by  the  high-level  design. 

•  Is  there  logical  consistency  within  models,  rules, 
knowledge,  information,  and  data? 

•  Does  redundancy  exist  within  models,  rules, 
knowledge,  information,  and  data? 

•  Are  uncertainties  propagated  correctly  within 
individual  modules? 

3.  Ensure  that  all  computer  code  conforms  to  standard 
style  and  documentation. 

4.  Ensure  that  all  system  programs  compile  and  operate 
without  errors. 

5.  Determine  if  the  system  is  complete  enough  to  cover 
all  realistic  parameters  and  situations. 
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6.  Survey  the  users  about  user-friendliness  of  the 

system. 

•  Does  the  user  interface  function  as  intended? 

•  Are  questions  asked  by  the  system  understandable? 

•  Is  the  output  understandable? 

•  Are  there  any  annoyances  to  the  user  in  the 
system? 

7.  Document  metadata  for  the  system  and  related 

databases. 

8.  Ensure  that  the  most  appropriate  data  for  the  system 

are  used. 

For  validation,  empirical  testing  of  a  DSS  using 
sound,  experimental  research  methods  is  always  best. 
Whole  system  testing  and  individual  component  testing 
are  both  necessary.  Again,  the  work  of  Geissman  and 
Schultz  (1988),  Cohen  and  Howe  (1989),  Wallace  and 
Fujii  (1989),  Grogono  and  others  (1991),  Sage  (1991), 
Adelman  (1992),  and  Finlay  (1994)  all  provide  insights 
into  empirical  methods  appropriate  for  validation. 
Suggesting  specific  steps  to  follow  for  system  valida¬ 
tion  is  difficult  because  system  performance  must  be 
measured  in  terms  of  how  the  decision-making  process 
has  been  affected,  and  often  the  process  may  have 
evolved  since  the  start  of  the  project.  However,  at  least 
nine  considerations  are  important  for  validation: 

1 .  Determine  if  each  module  or  component  makes  a 
contribution  to  improving  the  decision  maker’s 
performance. 

2.  Assess  whether  each  module  accurately  and  with 
minimum  bias  provides  an  assessment  of  the 
uncertainty  associated  with  the  module  output. 

3.  Evaluate  whether  individual  modules  represent 
realistic  descriptions  of  their  respective  environ¬ 
ments  as  well  as  average  and  extreme  cases. 

4.  Assess  whether  the  system  provides  a  realistic 
ecological  understanding  and  natural  resource 
management  perspective.  How  does  the  overall 
system  deal  with  average  cases  as  well  as  with 
extreme  cases? 

5.  Make  certain  assumptions  and  constraints  of  the 
system  are  clear  to  the  user. 

6.  Prior  to  beginning  system  development,  generate 
expectations  of  how  the  completed  system  will 
affect  decision  performance.  Design  evaluation 
processes  to  test  those  hypotheses. 

7.  Conduct  empirical  user  satisfaction  studies. 

8.  Evaluate  flexibility  of  the  system  to  respond  to 
changing  user  requirements. 

9.  Determine  the  reliability  of  measures  to  avoid 
ecologically  catastrophic  recommendations. 


Summary 


This  document  presented  a  broad  range  of  concepts, 
issues,  and  recommendations  related  to  the  need  for  and 
the  development  and  use  of  decision  support  systems. 
We  hope  that  it  will  be  useful  to  managers  and  decision 
makers,  as  well  as  potential  developers,  implementers, 
and  others  generally  interested  in  the  subject  of  DSS.  A 
DSS  was  generally  defined  as  an  interactive,  computer- 
based  tool  or  collection  of  tools  that  use(s)  information 
and  models  to  improve  the  process  and  outcome  of 
decision  making.  The  characteristics  and  capabilities  of 
DSS  were  discussed — what  a  DSS  is  and  is  not.  We  also 
caution  that  it  is  easy  to  develop  unrealistic  expectations 
of  a  DSS,  and  it  should  be  understood  that  a  DSS  is  not 
designed  to  replace  decision  makers,  nor  will  it  elimi¬ 
nate  poor  decisions. 

The  importance  of  meeting  resource  manager’s  needs 
and  other  customer  requirements  was  emphasized,  as 
was  the  need  to  develop  systems  that  provide  support 
for  specific  applications.  Providing  a  DSS  that  has  the 
desired  capabilities  but  is  not  usable  by  the  manager  is, 
at  best,  a  wasted  effort  and  at  worst  it  can  discourage 
managers  from  use  of  DSS  in  the  future.  The  reverse 
situation  (i.e.,  a  DSS  easily  used  by  managers  but  does 
not  have  the  capabilities  to  meet  application  needs)  is 
also  just  as  detrimental.  We  explain  why  information 
and  technology  transfer,  user  support,  and  other  matters 
related  to  institutionalization  of  DSS  are  issues  that 
need  to  be  addressed  in  order  to  achieve  successful 
long-term  development,  implementation,  and  use  of 
DSS. 

A  description  of  the  stages  in  the  decision-making 
process  is  presented,  as  is  how  this  process  should  aid 
in  framing  the  decision  process  of  natural  resource 
managers  and  determining  DSS  needs.  Although 
numerous  approaches  for  developing  a  DSS  exist,  we 
recommend  a  DSS  research  and  development  process 
that  recognizes  the  various  stages  of  the  decision¬ 
making  process  and  incorporates  evolutionary 
prototyping. 

We  stress  the  need  for  validation  and  verification 
when  evaluating  DSS  development  efforts.  An  overview 
of  validation  and  verification  methods  is  presented,  and 
several  steps  and  considerations  are  recommend  for 
performing  validation  and  verification  of  a  DSS.  The 
ultimate  goals  related  to  the  validation  and  verification 
of  decision  support  systems  are  to  “build  the  right 
systems  and  build  the  systems  right.” 
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For  those  who  want  additional  information  about 
various  aspects  of  DSS,  such  as  the  decision  process, 
collaborative  tools  for  decision  support,  information 
management  and  interoperability,  knowledge  manage¬ 
ment  and  decision  support,  and  computation,  communi¬ 
cation,  and  data  storage,  see  Appendix  2  (“Decision 
Support  Capabilities  for  Future  Technology  Require¬ 
ments,”  edited  by  Gene  Lessard). 
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The  glossary  is  used  with  permission  from  Powery 
D.J.,  1999,  Decision  Support  Systems  Glossary, 
Decision  Support  Systems  Resources:  World  Wide 
Web  http://DSSResources.com/glossary  accessed  on 
9  May  2001 .  Words  preceded  by  an  asterisk  have 
been  added  by  the  authors  of  this  document. 

Ad-Hoc  Query  -  Any  spontaneous  or  unplanned 
question  or  query.  It  is  a  query  that  consists  of 
dynamically  constructed  SQL,  which  is  usually 
constructed  by  desktop-resident  query  tools. 

Ad-Hoc  Query  Tool  -  An  end-user  tool  that  accepts  an 
English-like  or  point-and-click  request  for  data  and 
constructs  an  ad-hoc  query  to  retrieve  the  desired 
data  from  a  database. 

Agents  -  Self-contained  processes  that  run  in  the 
background  on  a  client  or  server  and  that  perform 
useful  functions  for  a  specific  user/owner.  Agents 
may  monitor  exceptions  based  on  criteria  or  execute 
automated  tasks.  For  example,  once  an  event  occurs 
a  daemon  performs  a  pre-defined  action  and  then  it 
returns  to  a  monitoring  state.  See  demon  or  daemon. 
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Aggregate  or  Aggregated  Data  -  Data  that  result  from 
applying  a  process  to  combine  data  elements.  Data 
that  are  summarized. 

Alerts  -  A  notification  from  an  event  that  a  trigger  has 
exceeded  a  predefined  threshold.  See  agents. 

Analytical  Hierarchy  Process  -  An  approach  to 
decision  making  that  involves  structuring  multiple 
choice  criteria  into  a  hierarchy,  assessing  the  relative 
importance  of  these  criteria,  comparing  alternatives 
for  each  criterion,  and  determining  an  overall 
ranking  of  the  alternatives. 

*Bayesian  Belief  Network  -  A  Bayesian  belief  network 
is  a  graphical  model  of  a  problem  that  represents 
causal  relations  and  is  used  for  calculating  probabil¬ 
ity  distributions  of  unobserved  variables  given  the 
observed  variables.  For  a  more  detailed  treatment, 
see  Jensen  (1996)  and  Pearl  (1988). 

Business  Data  -  Data  about  people,  places,  things, 
business  rules,  and  events  used  to  operate  a  business. 
It  is  not  metadata. 

Business  Intelligence  -  BI  is  a  popularized,  umbrella 
term  introduced  by  Howard  Dresner  of  the  Gartner 
Group  in  1989  to  describe  a  set  of  concepts  and 
methods  to  improve  business  decision  making  by 
using  fact-based  support  systems.  The  term  is 
sometimes  used  interchangeably  with  briefing  books 
and  executive  information  systems.  A  business 
intelligence  system  is  a  DSS. 

Business  Model  -  In  a  data  warehouse  it  is  the 

designer’s  view  of  how  the  business  functions.  The 
view  can  be  from  a  process,  data,  event,  or  resource 
perspective  and  can  be  the  past,  present,  or  future 
state  of  the  business. 

Business  Transaction  -  According  to  Microstrategy,  it 
is  a  unit  of  work  acted  upon  by  a  data  capture  system 
to  create,  modify,  or  delete  business  data.  Each 
transaction  represents  a  single  valued  fact  describing 
a  single  business  event. 

Client/Server  Architecture  -  A  network  architecture  in 
which  computers  on  a  network  act  as  a  server 
managing  files  and  network  services  OR  as  a  client 
where  users  run  applications  and  access  servers. 
Clients  rely  on  servers  for  resources  like  web  pages, 
data,  files,  printing,  and  On-line  Analytical  Process¬ 
ing  (OLAP). 

Cognitive  Overload  -  A  psychological  phenomenon 
characterized  by  an  overload  of  information  for  a 
decision  maker.  The  amount  of  information  exceeds 
the  person’s  cognitive  capacity.  DSS  can  reduce  or 
increase  cognitive  overload. 

Computer-Mediated  Communication  -  The  use  of 
computers  to  create,  store,  deliver,  and  process 
communications. 


Computer  Supported  Cooperative  Work  -  The  use  of 

computers  to  support  cooperative  work  among 
multiple  participants  (e.g.,  collaborative  authoring), 
as  distinct  from  work  that  may  not  be  cooperative. 

Conferencing,  Videoconferencing  or  Teleconferenc¬ 
ing  -  Real-time,  two-way  communications.  Audio¬ 
video  telecommunication  support  of  simultaneous 
interactions  among  participants  (e.g.,  involving 
conference  calls  or  videoconferencing). 

*  Constraint  Satisfaction  Programming  -  A  method 
for  solving  a  set  of  mathematical  problems  which 
uses  the  process  of  assigning  values  to  variables 
while  meeting  certain  requirements  or  “constraints.” 
It  is  a  form  of  artificial  intelligence  that  is  often  used 
in  the  field  of  optimization.  Some  common  uses  are 
scheduling,  manufacturing,  and  planning.  It  has  a 
number  of  benefits  (mainly  flexibility)  over  other 
common  approaches  to  these  problems,  like  linear 
programming. 

Controllable  Variables  -  Decision  variables  that  can  be 
changed  and  manipulated  by  a  decision  maker,  such 
as  quantity  to  produce,  amount  of  resources  to 
allocate,  etc. 

Corporate  Planning  System  -  A  decision  support 
system  that  holds  and  derives  knowledge  relevant  to 
planning  decisions  that  cut  across  organizational 
units  and  involve  all  of  an  organization’s  functions 
(i.e.,  its  operations,  finance,  marketing,  personnel, 
etc.). 

Critical  Success  Factors  -  Key  areas  of  business 

activity  in  which  favorable  results  are  necessary  for  a 
company  to  reach  its  goals. 

Data  -  Binary  (digital)  representations  of  atomic  facts, 
text,  graphics,  bit-mapped  images,  sound,  analog  or 
digital  live-video  segments.  Data  is  the  raw  material 
of  a  system  supplied  by  data  producers  and  is  used 
by  information  consumers  to  create  information. 

Data  Conferencing  -  This  term  refers  to  a  communica¬ 
tion  session  in  which  two  or  more  participants  are 
sharing  computer-based  data  in  real-time.  Any 
participants’  keyboard/mouse  can  control  screens  of 
other  participants.  Voice  communication  can  be  out- 
of-band  using  a  totally  separate  voice  connection  or 
in-band  using  a  simultaneous  voice  and  data  technol¬ 
ogy. 

Data  Dictionary  -  A  database  about  data  and  database 
structures.  A  catalog  of  all  data  elements,  containing 
their  names,  structures,  and  information  about  their 
usage.  A  central  location  for  metadata.  Normally, 
data  dictionaries  are  designed  to  store  a  limited  set  of 
available  metadata,  concentrating  on  the  information 
relating  to  the  data  elements,  databases,  files  and 
programs  of  implemented  systems. 
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Data-Driven  DSS  or  Data-Oriented  DSS  -  This  type 
of  DSS  emphasizes  access  to  and  manipulation  of  a 
time-series  of  internal  company  data  and  sometimes 
external  data.  Simple  file  systems  accessed  by  query 
and  retrieval  tools  provide  the  most  elementary  level 
of  functionality.  Data  warehouse  systems  that  allow 
the  manipulation  of  data  by  computerized  tools 
tailored  to  a  specific  task  and  setting  or  by  more 
general  tools  and  operators  provide  additional 
functionality.  Data-driven  DSS  with  OLAP  or  data 
mining  tools  provide  the  highest  level  of  functional¬ 
ity  and  decision  support  that  is  linked  to  analysis  of 
large  collections  of  historical  data.  Early,  very 
limited  versions  of  data-driven  DSS  were  called 
Retrieval-Only  DSS  by  Bonczek  and  others  (1981). 

Data  Element  -  The  most  elementary  unit  of  data  that 
can  be  identified  and  described  in  a  dictionary  or 
repository  which  cannot  be  subdivided. 

Data  Mining  -  A  class  of  analytical  applications  that 
search  for  hidden  patterns  in  a  data  base.  Data 
mining  is  the  process  of  sifting  through  large 
amounts  of  data  to  produce  data  content  relation¬ 
ships.  This  is  also  known  as  data  surfing.  Data 
mining  tools  use  a  variety  of  techniques  including 
case-based  reasoning,  data  visualization,  fuzzy  query 
and  analysis,  and  neural  networks.  Case-based 
reasoning  tools  provide  a  means  to  find  records 
similar  to  a  specified  record  or  records.  These  tools 
let  the  user  specify  the  “similarity”  of  retrieved 
records.  Data  visualization  tools  let  the  user  easily 
and  quickly  view  graphical  displays  of  information 
from  different  perspectives. 

Data  Quality  -  High  quality  data  is  accurate,  timely, 
meaningful,  and  complete.  DSS  must  have  high 
quality  data;  low  quality  data  can  result  in  bad 
decisions.  Assessing  or  measuring  data  quality  is  a 
preliminary  task  associated  with  evaluating  the 
feasibility  of  a  data-driven  DSS  project. 

Data  Warehouse  -  A  database  designed  to  support 
decision  making  in  organizations.  It  is  batch  updated 
and  structured  for  rapid  online  queries  and  manage¬ 
rial  summaries.  Data  warehouses  contain  large 
amounts  of  data.  A  data  warehouse  is  a  subject- 
oriented,  integrated,  time-variant,  nonvolatile 
collection  of  data  in  support  of  management’s 
decision  making  process.  Check  “What  is  a  Data 
Warehouse”  by  W.H.  Inmon  at  http:// 
www.cait.wustl.edu/cait/papers/prism/voll_nol/. 
According  to  Kimball  (1996),  “a  data  warehouse  is  a 
copy  of  transaction  data  specifically  structured  for 
query  and  analysis”  (see  “A  Definition  of  Data 
Warehousing”  by  I.  Greenfield  at  http:// 
pwp.starnetinc.com/larryg/defmed.html.) 


Data  Visualization  -  This  term  refers  to  presenting  data 
and  summary  information  using  graphics,  animation, 
3-D  displays,  and  other  multimedia  DSS  tools. 

Decision  -  The  choice  of  one  from  among  a  number  of 
alternatives;  a  statement  indicating  a  commitment  to 
a  specific  course  of  action. 

Decision  Analysis  Tools  -  DA  tools  help  decision 
makers  decompose  and  structure  problems.  The  aim 
of  these  tools  is  to  help  a  user  apply  models  like 
decision  trees,  multi-attribute  utility  models,  Baye¬ 
sian  models,  Analytical  Hierarchy  Process  (AHP), 
etc.  Examples  of  DA  software  packages  include 
AliahThink,  BestChoice3,  Criterium  Decision  Plus, 
DecideRight,  DecisionMaker,  Demos,  DPL,  Expert 
Choice,  Strad,  Supertree,  and  Which  and  Why. 

Decision  Room  -  A  physical  arrangement  for  a  group 
DSS  in  which  workstations  are  available  to  partici¬ 
pants.  The  objective  for  using  a  Decision  Room  is  to 
enhance  and  improve  the  group’s  decision-making 
process. 

Decision  Systems  are  computer  based  programs  and 
technologies  intended  to  make  routine  decisions, 
monitor  and  control  processes,  and  aid  or  assist 
decision  makers  in  semi-structured  and/or  non¬ 
routine  decision  situations. 

Decision  Support  Systems  (DSS)  are  interactive 
computer-based  systems  intended  to  help  decision 
makers  utilize  data  and  models  to  identify  and  solve 
problems  and  make  decisions.  The  “system  must  aid 
a  decision  maker  in  solving  unprogrammed,  unstruc¬ 
tured  (or  ‘semistructured’)  problems... the  system 
must  possess  an  interactive  query  facility,  with  a 
query  language  that  ...is  ...easy  to  learn  and  use” 
(Bonczek  and  others,  1981:19).  DSS  help  managers/ 
decision  makers  use  and  manipulate  data;  apply 
checklists  and  heuristics;  and  build  and  use  math¬ 
ematical  models.  According  to  Turban  (1990),  a  DSS 
has  four  major  characteristics:  DSS  incorporate  both 
data  and  models;  they  are  designed  to  assist  manag¬ 
ers  in  their  decision  processes  in  semi  structured  (or 
unstructured)  tasks;  they  support,  rather  than  replace, 
managerial  judgment;  and  their  objective  is  to 
improve  the  effectiveness  of  the  decisions,  not  the 
efficiency  with  which  decisions  are  being  made. 

Decision  Variables  -  In  a  model-driven  DSS  a  decision 
variable  is  a  changing  factor  in  the  model  that  is 
determined  by  a  decision  maker.  They  are  sometimes 
called  independent  variables,  and  the  range  of  values 
for  the  decision  variables  constrain  the  choices  of  the 
decision  maker. 

Demon  or  Daemon  -  A  computer  program  or  proce¬ 
dure  that  is  automatically  activated  when  it  recog¬ 
nizes  a  specific,  predefined  state  or  condition. 
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Descriptive  Model  -  Physical,  conceptual,  or  math¬ 
ematical  models  that  describe  situations  as  they  are 
or  as  they  actually  appear. 

Deterministic  Model  -  Mathematical  models  that  are 
constructed  for  a  condition  of  assumed  certainty.  The 
models  assume  there  is  only  one  possible  result 
(which  is  known)  for  each  alternative  course  or 
action. 

Development  Environment  -  The  DE  is  used  by  a 
designer/builder.  A  development  environment 
typically  includes  software  for  creating  and  main¬ 
taining  a  knowledge  base  and  software  for  the 
inference  engine. 

Dialog  Generation  and  Management  System 
(DGMS)  -  A  software  management  package  in  a 
DSS  whose  functions  in  the  dialog  subsystem  is 
similar  to  that  of  a  DBMS  in  a  database  (see  Sprague 
and  Carlson,  1982). 

Dialog  System  -  The  hardware  and  software  that  create 
and  implement  a  user  interface  for  a  DSS.  A  DSS 
dialog  system  creates  the  human-computer  interface. 

Domain  Expert  -  A  person  who  has  expertise  in  the 
domain  in  which  a  specific  expert  system  is  being 
developed.  A  domain  expert  works  closely  with  a 
developer  (known  as  a  knowledge  engineer)  to 
capture  the  expert’s  knowledge  (especially  rule  and 
relationship  information)  in  a  computer  readable 
representation  often  called  a  knowledge  base. 

Drill  Down/Up  -  An  analytical  technique  that  lets  a 
DSS  user  navigate  among  levels  of  data  ranging 
from  the  most  summarized  (up)  to  the  most  detailed 
(down). 

DSS  Generator  -  Computer  software  package  that 
provides  tools  and  capabilities  that  help  a  developer 
quickly  and  easily  build  a  specific  Decision  Support 
System  (see  Sprague  and  Carlson,  1982:1 1).  Excel  is 
an  example  of  a  DSS  Generator.  Many  companies 
market  tools  for  building  DSS  and  EIS. 

DSS  Development  Tools  -  Software  components  (such 
as  editors,  code  libraries,  specific  objects,  visual 
interfaces)  that  facilitate  the  development  of  a 
specific  DSS. 

e-Meetings  -  A  term  for  a  meeting  supported  by  full- 
motion  video,  audio,  and  web  meeting  tools.  One  or 
more  participants  in  the  meeting  is  participating 
remotely  in  the  meeting.  It  is  possible  that  all 
participants  are  in  different  physical  locations. 

Enterprise- wide  DSS  -  A  DSS  that  supports  a  large 
group  of  managers  in  a  networked  client-server 
environment  with  a  specialized  data  warehouse  as 
part  of  the  DSS  architecture. 

Evolutionary  (Iterative)  Design  Process  -  A  system¬ 
atic  process  for  system  development  that  is  recom¬ 
mended  for  use  in  creating  DSS.  A  portion  of  the 


DSS  system  is  quickly  constructed,  then  tested, 
improved,  and  enlarged  in  systematic  steps.  This 
methodology  is  similar  to  prototyping. 

Exception  Reporting  -  A  reporting  philosophy  and 
approach  that  supports  Management  by  Exception. 
Reports  should  be  designed  to  display  significant 
exceptions  in  results  and  data.  The  idea  is  to  “flag” 
important  information  and  bring  it  quickly  to  the 
attention  of  managerial  users  of  the  report.  Exception 
reporting  can  be  implemented  in  any  type  of  DSS, 
but  it  is  particularly  useful  in  data-driven  DSS  and 
EIS. 

Executive  Information  Systems  (EIS)  -  A  computer¬ 
ized  system  intended  to  provide  current  and  appro¬ 
priate  information  to  support  executive  decision 
making  for  managers  using  a  networked  workstation. 
The  emphasis  is  on  graphical  displays  and  an  easy  to 
use  interface  that  present  information  from  the 
corporate  database.  They  are  tools  to  provide  canned 
reports  or  briefing  books  to  top-level  executives. 

They  offer  strong  reporting  and  drill-down  capabili¬ 
ties. 

Executive  Support  Systems  (ESS)  -  An  executive 
information  system  (EIS)  that  includes  specific 
decision  aiding  and/or  analysis  capabilities. 

Expert  Systems  -  Are  man-machine  systems  with 
specialized  problem-solving  expertise.  The  “exper¬ 
tise”  consists  of  knowledge  about  a  particular 
domain,  understanding  of  problems  within  that 
domain,  and  “skill”  at  solving  some  of  these  prob¬ 
lems. 

Facilitator  -  A  person(s)  who  manages  the  use  of  a 
group  decision  support  system  from  initial  planning 
through  actual  operation. 

Feasibility  Study  -  A  study  of  the  technical  and 

economic  prospects  for  developing  a  system  prior  to 
actually  committing  resources  to  actually  developing  it. 

Functional  DSS  -  A  decision  support  system  that  holds 
and  derives  knowledge  relevant  for  decisions  about 
some  function  an  organization  performs  (e.g.,  a 
marketing  function,  a  production  function). 

Generators  -  Software  packages  that  are  designed  to 
expedite  programming  efforts  that  are  required  to 
build  information  systems,  especially  expert  and 
decision  support  systems. 

Goal-seeking  -  The  capability  of  asking  the  computer 
software  what  values  certain  variables  must  have  in 
order  to  attain  desired  goals.  It  is  a  tool  that  uses 
iterative  calculations  to  find  the  value  required  in  one 
cell  (variable)  in  order  to  achieve  a  desired  value  in 
another  cell.  A  common  use  of  the  goal-seeking 
feature  in  a  spreadsheet  is  calculating  a  break-even 
quantity. 
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Geographic  Information  Systems  (GIS)  -  A  support 
system  that  represents  data  using  maps.  It  helps 
people  access,  display  and  analyze  data  that  have 
geographic  content  and  meaning.  Check  U.S. 
Geological  Survey  page  on  geographic  information 
systems  at  http://www.usgs.gov/research/gis/ 
title.html.  Examples  of  software  packages  include 
Arc  View,  Map/IDIS,  Proximity,  and  TargetView. 

Graphical  User  Interface  (GUI)  -  A  program  interface 
that  uses  a  computer’s  graphics  capabilities  to  make 
the  program  easier  to  use.  Graphical  interfaces  use  a 
pointing  device  to  select  objects,  including  icons, 
menus,  text  boxes,  etc.  A  GUI  includes  standard 
formats  for  representing  text  and  graphics. 

Group  Decision  Support  Systems  (GDSS)  -  An 
interactive,  computer-based  system  that  facilitates 
solution  of  unstructured  problems  by  a  set  of 
decision  makers  working  together  as  a  group.  It  aids 
groups,  especially  groups  of  managers,  in  analyzing 
problem  situations  and  in  performing  group  decision¬ 
making  tasks. 

Groupware  -  Is  software  designed  to  support  more  than 
one  person  working  on  a  shared  task.  Groupware  is 
an  evolving  concept  that  is  more  than  multiuser 
software  which  allows  access  to  the  same  data. 
Groupware  provides  a  mechanism  that  helps  users 
coordinate  and  keep  track  of  on-going  projects.  It 
allows  people  to  work  together  through  computer- 
supported  communication,  collaboration,  and 
coordination.  Lotus  Notes,  Microsoft  Exchange, 
Communicator,  Novell  GroupWise,  Netscape 
SuiteSpot,  Eclipse,  Team  Talk,  and  Internet  Ex- 
plorer/NetMeeting  are  examples  of  groupware 
products. 

Heuristics  -  The  informal,  judgmental  knowledge  of  an 
application  area  that  constitutes  the  “rules  of  good 
judgment”  in  the  field.  Heuristics  also  encompass  the 
knowledge  of  how  to  solve  problems  efficiently  and 
effectively,  how  to  plan  steps  in  solving  a  complex 
problem,  how  to  improve  performance,  and  so  forth. 
From  the  Greek  word  “Heuriskein”  meaning  “to 
discover” 

Hypermedia  -  Combination  of  several  types  of  media 
such  as  text,  graphics,  audio,  and  video. 

Hypertext  -  An  approach  for  handling  text  and  other 
information  that  allows  users  to  jump  from  a  given 
topic,  whenever  he  or  she  wishes,  to  related  topics.  A 
knowledge  management  technique  in  which  knowl¬ 
edge  is  represented  in  linked  documents  and  pro¬ 
cessed  in  a  way  that  allows  a  user  to  select  a  high¬ 
lighted  marker  on  the  currently  viewed  page  and 
access  a  linked  page  about  a  topic  indicated  by  the 
marker. 


Icon  -  A  visual,  graphic  representation  of  an  object, 
word,  or  concept. 

Independent  Variables  -  Variables  in  a  model  that  are 
controlled  by  the  environment  and  that  influence  the 
results  of  a  decision  (also  called  Input  Variables, 
parameters,  givens). 

Inference  -  The  process  of  drawing  a  conclusion  from 
given  evidence.  To  reach  a  decision  by  reasoning. 

Inference  Engine  -  That  part  of  an  expert  system  that 
actually  performs  the  reasoning  function. 

Information  -  Data  that  has  been  processed  to  add  or 
create  meaning  and  hopefully  knowledge  for  the 
person  who  receives  it.  Information  is  the  output  of 
information  systems. 

Information  Economics  -  This  term  refers  to  an 
approach  to  evaluating  DSS/IS  projects  using  a 
scoring  approach  to  cost/benefit  analysis  that 
assesses  technical  and  company  tangible  and 
intangible  benefits  and  costs  (see  Parker  and  others, 
1989). 

Information  Systems  Architecture  -  A  formal  defini¬ 
tion  of  the  business  processes  and  rules,  systems 
structure,  technical  framework,  and  product  tech¬ 
nologies  for  business  information  systems.  An 
information  systems  architecture  consists  of  four 
layers:  business  process  architecture,  systems 
architecture,  technical  architecture,  and  product  delivery 
architecture. 

Interdependent  Decisions  -  A  series  of  decisions  that 
are  interrelated.  A  sequential  set  of  decisions  are 
usually  interdependent. 

Internet  -  The  Internet  (capitalized)  refers  specifically 
to  the  DARPA  Internet  and  the  TCP/IP  protocols  it 
uses.  The  Internet  is  a  collection  of  packet-switching 
networks  and  routers  that  uses  the  TCP/IP  protocol 
suit  and  functions  as  a  single,  cooperative  virtual 
network.  It  is  a  global  web  connecting  more  than  one 
million  computers. 

^Interoperability  -  The  capability  to  communicate, 
execute  programs,  or  transfer  data  among  various 
functional  units  in  a  manner  that  requires  the  user  to 
have  little  or  no  knowledge  of  the  unique  characteris¬ 
tics  of  those  units  (ISO  TC21 1,  specifically,  ISO 
191 18  -  Geographic  information  -  Encoding).  Or  in 
more  general  terms,  the  ability  to  seamlessly  and 
transparently  “plug  and  play.” 

Intranet  -  An  internal  organizational  network  using 
TCP/IP  with  at  least  one  web  server  that  is  only 
accessible  by  an  organization’s  members  or  others 
who  have  specific  authorization.  A  firewall  and 
password  protection  limit  access  to  the  network.  The 
intranet  is  used  to  share  corporate  information, 
including  DSS  capabilities.  See  web-based  DSS  at 
http://dss.resources.com/dss/online.html  and  check 
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the  Intranet  FAQ  at  http://www.intrack.com/intranet/ 
ifaq.shtml. 

Knowledge  -  Knowledge  refers  to  what  one  knows  and 
understands.  Knowledge  is  sometimes  categorized  as 
either  unstructured,  structured,  explicit  or  tacit.  What 
we  know  we  know  is  explicit  knowledge.  Knowledge 
that  is  unstructured  and  understood,  but  not  clearly 
expressed  is  implicit  knowledge.  If  the  knowledge  is 
organized  and  easy  to  share  then  it  is  called  struc¬ 
tured  knowledge.  To  convert  implicit  knowledge  into 
explicit  knowledge,  it  must  be  extracted  and  format¬ 
ted. 

Knowledge  Acquisition  -  The  extraction  and  formula¬ 
tion  of  knowledge  derived  from  various  sources, 
especially  from  experts. 

Knowledge  Base  -  A  collection  of  facts,  rules,  and 
procedures  organized  into  schemas.  The  assembly  of 
all  the  information  and  knowledge  of  a  specific  field 
of  interest. 

Knowledge  Engineer  -  An  AI  specialist  responsible  for 
the  technical  side  of  developing  an  expert  system. 
The  knowledge  engineer  works  closely  with  the 
domain  expert  to  capture  the  expert’s  knowledge  in  a 
knowledge  base. 

Knowledge  Engineering  (KE)  -  The  engineering 
discipline  that  involves  integrating  knowledge  into 
computer  systems  in  order  to  solve  complex  prob¬ 
lems  normally  requiring  a  high  level  of  human 
expertise. 

Knowledge  Management  (KM)  -  KM  is  the  distribu¬ 
tion,  access,  and  retrieval  of  unstructured  informa¬ 
tion  about  “human  experiences”  between  interdepen¬ 
dent  individuals  or  among  members  of  a  workgroup. 
Knowledge  management  involves  identifying  a 
group  of  people  who  have  a  need  to  share  knowl¬ 
edge,  developing  technological  support  that  enables 
knowledge  sharing,  and  creating  a  process  for 
transferring  and  disseminating  knowledge. 

Knowledge  Management  Software  (KMS)  -  Software 
that  can  store  and  manage  unstructured  information 
in  a  variety  of  electronic  formats.  The  software  may 
assist  in  knowledge  capture,  categorization,  deploy¬ 
ment,  inquiry,  discovery,  or  communication.  Prod¬ 
ucts  include  electronic  document  management 
systems  (EDMS).  Visit  KMWorld  at  http:// 
www.kmworld.com/. 

Linear  Programming  -  A  mathematical  model  for 
optimal  solution  of  resource  allocation  problems. 

Metadata  or  Meta  Data  -  Data  about  the  data  in  a  data 
warehouse.  Metadata  provides  a  directory  to  help  the 
DSS  locate  the  contents  of  the  data  warehouse;  it  is  a 
guide  to  mapping  data  as  it  is  transformed  from  the 
operational  environment  to  the  data  warehouse 
environment;  and  it  serves  as  a  guide  to  the 


algorithms  used  for  summarization  of  current 
detailed  data.  Metadata  is  semantic  information 
associated  with  a  given  variable.  Metadata  must 
include  business  definitions  of  the  data  and  clear, 
accurate  descriptions  of  data  types,  potential  values, 
original  source  system,  data  formats,  and  other 
characteristics.  Metadata  defines  and  describes 
business  data.  Examples  of  metadata  include  data 
element  descriptions,  data  type  descriptions,  at¬ 
tribute/property  descriptions,  range/domain  descrip¬ 
tions,  and  process/method  descriptions.  The  reposi¬ 
tory  environment  encompasses  all  corporate 
metadata  resources:  database  catalogs,  data  dictio¬ 
naries,  and  navigation  services.  Metadata  includes 
things  like  the  name,  length,  valid  values,  and 
description  of  a  data  element.  Metadata  is  stored  in  a 
data  dictionary  and  repository.  It  insulates  the  data 
warehouse  from  changes  in  the  schema  of  opera¬ 
tional  systems. 

Methodology  -  A  system  of  principles,  practices,  and 
procedures  applied  to  a  specific  branch  of  knowl¬ 
edge. 

Middleware  -  A  communications  layer  that  allows 
applications  to  interact  across  hardware  and  network 
environments. 

Model  Base  -  A  collection  of  preprogrammed  quantita¬ 
tive  models  (e.g.,  statistical,  financial,  optimization) 
organized  as  a  single  unit. 

Model-driven  DSS  or  Model-oriented  DSS  -  This  type 
of  DSS  emphasizes  access  to  and  manipulation  of  a 
model,  e.g.,  statistical,  financial,  optimization  and/or 
simulation.  Simple  statistical  and  analytical  tools 
provide  the  most  elementary  level  of  functionality. 
Some  OLAP  systems  that  allow  complex  analysis  of 
data  may  be  classified  as  hybrid  DSS  systems 
providing  both  modeling  and  data  retrieval  and  data 
summarization  functionality.  Data  mining  is  also  a 
hybrid  approach  to  DSS.  In  general,  model-driven 
DSS  use  complex  financial,  simulation,  optimization 
and/or  rule  (expert)  models  to  provide  decision 
support.  Model-driven  DSS  use  data  and  parameters 
provided  by  decision  makers  to  aid  decision  makers 
in  analyzing  a  situation,  but  they  are  not  usually  data 
intensive,  that  is  very  large  data  bases  are  usually  not 
needed  for  model-driven  DSS.  Early  versions  of 
model-driven  DSS  were  called  Computationally 
Oriented  DSS  by  Bonczek  and  others  (1981). 

Modeling  Tools  -  Software  programs  that  help  develop¬ 
ers  and  users  build  mathematical  models  quickly. 
Spreadsheets  and  planning  languages  like  IFPS  are 
modeling  tools. 

Multidimensional  Database  (MDBS  and  MDBMS)  - 

A  database  that  lets  users  analyze  large  amounts  of 
data.  An  MDBS  captures  and  presents  data  as  arrays 
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that  can  be  arranged  in  multiple  dimensions.  Vari¬ 
ables  are  the  objects  that  hold  data  in  a  multidimen¬ 
sional  database.  These  are  simply  arrays  of  values 
(usually  numeric)  that  are  “dimensioned”  by  the 
dimensions  in  a  database.  For  example,  a  UNITS 
variable  may  be  dimensioned  by  MONTH,  PROD¬ 
UCT,  and  REGION.  This  three-dimensional  variable 
or  array  is  often  visualized  as  a  cube  of  data. 
Multidimensional  databases  can  have  multiple 
variables,  with  common  or  a  unique  set  of  dimen¬ 
sions.  This  multidimensional  view  of  data  is  espe¬ 
cially  powerful  for  OLAP  applications. 

Multiparticipant  DSS  -  A  decision  support  system  that 
supports  multiple  participants  engaged  in  a  decision¬ 
making  task  (or  functions  as  one  of  the  participants). 
See  Group  DSS. 

Multipoint  Conference  -  An  audio,  data  and/or  video 
conference  among  more  than  two  remote  partici¬ 
pants. 

Multipoint  Control  Unit  (MCU)  -  A  device  used  to 
link  remote  sites  into  a  single  conference  call  or  a 
device  to  manage  several  simultaneous,  independent 
conferences. 

Normalization  -  The  process  of  reducing  a  complex 
data  structure  into  its  simplest,  most  stable  structure. 
In  general,  the  process  entails  the  removal  of 
redundant  attributes,  keys,  and  relationships  from  a 
conceptual  data  model. 

Object  -  A  person,  place,  thing,  or  concept  that  has 
characteristics  of  interest  to  an  environment.  In  terms 
of  an  object-oriented  system,  an  object  is  an  entity 
that  combines  descriptions  of  data  and  behavior. 

On-line  Analytical  Processing  (OLAP)  -  Software  for 
manipulating  multidimensional  data  from  a  variety 
of  sources  that  has  been  stored  in  a  data  warehouse. 
The  software  can  create  various  views  and  represen¬ 
tations  of  the  data.  OLAP  software  provides  fast, 
consistent,  interactive  access  to  shared,  multidimen¬ 
sional  data.  Check  the  Guide  to  OLAP  Terminology 
from  the  OLAP  Council  at  http://dss.cba.uni.edu/ 
glossary/olaptrms.html. 

Operational  or  Transaction  Database  -  The  database- 
of-record  for  a  transaction-update  system.  The 
operational  database  is  the  source  of  data  for  the  data 
warehouse.  It  contains  detailed  data  used  to  run  the 
day-to-day  operations  of  the  business.  The  data 
continually  changes  as  updates  are  made  and  reflect 
the  current  value  of  the  last  transaction. 

Optimize  -  The  decision  strategy  of  choosing  the 
alternative  that  gives  the  best  or  optimal  overall 
value. 

Organizational  DSS  -  A  multiparticipant  DSS  designed 
to  support  a  decision  maker  in  a  setting  that  has  a 
more  elaborate  infrastructure  than  a  group  (i.e., 


involving  specialized  roles,  restricted  communication 
patterns,  differing  authority  levels).  See  Enterprise- 
Wide  DSS. 

Pivot  -  Changing  the  dimensional  orientation  of  a 
display  or  report.  See  “rotate”  in  the  OLAP  Guide  to 
terms  at  http://dss.cba.uni.edu/glossary/ 
olaptrms.html. 

Planning  -  A  managerial  function  concerned  with 
making  forecasts,  formulating  outlines  of  things  to 
do,  and  identifying  methods  to  accomplish  them. 

Prototyping  -  A  strategy  in  system  development  in 
which  a  scaled  down  system  or  portion  of  a  system  is 
constructed  in  a  short  time,  tested,  and  improved  in 
several  iterations.  A  prototype  is  an  initial  version  of 
a  system  that  is  quickly  developed  to  test  the 
effectiveness  of  the  overall  design  being  used  to 
solve  a  particular  problem.  Prototyping  is  similar  to 
the  Evolutionary  (Iterative)  Design  Process.  It  is 
sometimes  termed  rapid  prototyping  and  is  similar  to 
rapid  application  development  (RAD). 

Query  -  Generically,  query  means  question.  Usually  it 
refers  to  a  complex  SQL  SELECT  statement  for 
decision  support.  See  Ad-Hoc  Query. 

Rapid  Application  Development  (RAD)  -  Part  of  a 
methodology  that  specifies  incremental  development 
with  constant  feedback  from  the  customers.  The 
point  is  to  keep  projects  focused  on  delivering  value 
and  to  keep  clear  and  open  lines  of  communication. 
Oral  and  written  communication  is  not  completely 
adequate  for  specification  of  computer  systems. 

RAD  overcomes  the  limitations  of  language  by 
minimizing  the  time  between  concept  and  implemen¬ 
tation. 

Rational  Decision  Behavior  -Behavior  that  is  goal- 
oriented  in  reaching  a  decision.  Behavior  is  guided 
by  the  consequences  likely  to  result  from  the 
selection  of  a  given  alternative.  A  decision  maker 
believes  based  upon  analysis  that  a  chosen  alterna¬ 
tive  will  result  in  achieving  one  or  more  desired 
objectives.  Rational  decision  behavior  should  be 
supported  by  DSS. 

Record  -  A  group  of  data  values  consisting  of  one  value 
for  each  of  a  prescribed  set  of  relational  fields;  an 
occurrence  of  a  record  type. 

Report  and  Query  Tools  -  These  tools  produce  a 
tabular  list  of  information  from  data  stored  in  a 
relational  database.  Examples  include  Microsoft 
Access  and  Brio  Query. 

Representation  -  The  formulation  or  view  of  a  prob¬ 
lem.  Developed  so  the  problem  will  be  easier  to 
solve. 

Result  Variables  -  In  a  model-driven  DSS  a  result 
variable  shows  the  consequences  of  changing 
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decision  variables.  Result  variables  are  also  referred 
to  as  dependent  variables. 

ROMC  (Representation,  Operations,  Memory  Aids, 
Mechanism  Control)  Design  Approach  -  A 

systematic  approach  for  developing  large-scale  DSS, 
especially  user  interfaces.  It  is  a  user-oriented 
approach  for  stating  system  performance  require¬ 
ments  (see  Sprague  and  Carlson,  1982). 

Rule  -  A  formal  way  of  specifying  a  recommendation, 
directive,  or  strategy,  expressed  as  an  IF  premise 
THEN  conclusion. 

Scalability  -  The  ability  to  scale  hardware  and  software 
to  support  larger  or  smaller  volumes  of  data  and 
more  or  less  users.  The  ability  to  increase  or  de¬ 
crease  size  or  capability  in  cost-effective  increments 
with  minimal  impact  on  the  unit  cost  of  business  and 
the  procurement  of  additional  services. 

Semistructured  Decisions  -  Decisions  in  which  some 
aspects  of  the  problem  are  structured  and  others  are 
unstructured. 

Sensitivity  Analysis  -  Running  a  decision  model 
several  times  with  different  inputs  so  a  modeler  can 
analyze  the  alternative  results. 

Shell  -  An  expert  system  development  tool  consisting  of 
two  stand-alone  pieces  of  software:  a  rule  set 
manager  and  an  inference  engine  capable  of  reason¬ 
ing  with  rules  set  built  with  the  rule  set  manager.  A 
shell  is  a  complete  expert  system  stripped  of  its 
specific  knowledge. 

Simulation  -  A  technique  for  conducting  one  or  more 
experiments  that  test  various  outcomes  resulting 
from  a  quantitative  model  of  a  system. 

Specific  DSS  -  A  computer-based  system  that  actually 
helps  a  person  accomplish  a  specific  task.  “Specific 
DSS  are  the  hardware/software  that  allow  a  specific 
decision  maker  or  group  of  them  to  deal  with 
specific  sets  of  related  problems”  (see  Sprague  and 
Carlson,  1982:10). 

Spreadsheet  -  In  the  accounting  world  a  spreadsheet 
was  and  is  a  large  sheet  of  paper  that  lays  everything 
out  for  a  businessperson.  It  spreads  or  shows  all  of 
the  costs,  income,  taxes,  etc.,  on  a  single  sheet  of 
paper  for  a  manager  to  look  at  when  making  a 
decision.  An  electronic  spreadsheet  organizes 
information  into  columns  and  rows.  The  data  can 
then  be  “added  up”  by  a  formula  to  give  a  total  or 
sum.  The  spreadsheet  summarizes  information  from 
many  sources  in  one  place  and  presents  the  informa¬ 
tion  in  a  format  to  help  a  decision  maker  see  the 
financial  “big  picture”  for  the  company.  A  program 
that  has  a  collection  of  cells  whose  values  can  be 
displayed  on  a  computer  screen.  By  changing  cell 
definitions  and  having  all  cell  values  reevaluated,  a 
user  can  readily  observe  the  effects  of  those  changes. 


Decision  support  systems  built  using  spreadsheet 
software  are  sometimes  called  Spreadsheet  DSS.  See 
“A  Brief  History  of  Spreadsheets”  by  Daniel  Power 
at  http://dss.cba.uni.edu/dss/sshistory.htmk 

Star  Schema  -  A  relational  database  schema  organized 
around  a  central  table  (fact  table)  joined  to  a  few 
smaller  tables  (dimension  tables)  using  foreign  key 
references.  The  fact  table  contains  raw  numeric  items 
that  represent  relevant  business  facts  (price,  discount 
values,  number  of  units  sold,  dollar  value,  etc.).  The 
facts  are  typically  additive  and  are  accessed  via 
dimensions.  Since  the  fact  tables  are  presummarized 
and  aggregated  along  business  dimensions,  these 
tables  tend  to  be  very  large.  The  basic  premise  of  star 
schemas  is  that  information  can  be  classified  into 
two  groups:  facts  and  dimensions.  Facts  are  the  core 
data  element  being  analyzed.  For  example,  units  of 
individual  items  sold  are  facts,  while  dimensions  are 
attributes  about  the  facts.  Dimensions  are  the  product 
types  purchased  and  the  date  of  purchase.  The  star 
schema  has  also  been  called  a  star-join  schema,  data 
cube,  data  list,  grid  file,  and  multidimensional 
schema.  The  name  star  schema  comes  from  the 
pattern  formed  by  the  entities  and  relationships  when 
they  are  represented  as  an  entity-relationship 
diagram  (ERD).  The  results  of  a  business  activity  are 
at  the  center  of  the  star  surrounded  by  the  people, 
places,  and  things  that  come  together  to  perform  this 
activity.  These  dimensions  are  the  points  of  the  star. 

Strategic  Planning  -  A  decision-making  process  in 
which  decisions  are  made  about  establishing  organi¬ 
zational  purposes/missions,  determining  objectives, 
selecting  strategies,  and  setting  policies. 

Structured  Decisions  -  Standard  or  repetitive  decision 
situations  for  which  solution  techniques  are  already 
available  (also  sometimes  called  routine  or  pro¬ 
grammed  decisions).  The  structural  elements  in  the 
situation  (e.g.,  alternatives,  criteria,  environmental 
conditions)  are  known,  defined,  and  understood. 

Symbolic  Processing  -  Use  of  symbols,  rather  than 
numbers,  combined  with  rules-of-thumb  (or  heuris¬ 
tics),  in  order  to  process  information  and  solve 
problems. 

Systems  Development  Life  Cycle  (SDLC)  -  A  process 
by  which  systems  analysts,  software  engineers, 
programmers,  and  end-users  build  systems.  It  is  a 
project  management  tool,  used  to  plan,  execute,  and 
control  systems  development  projects.  The  steps  in 
the  cycle  include:  (1)  determine  user  requirements; 

(2)  systems  analysis;  (3)  overall  system  design;  (4) 
detailed  system  design;  (5)  programming;  (6)  testing; 
and  (7)  implementation.  Each  step  is  concluded  by 
developing  a  written  document  that  must  be  re¬ 
viewed  and  approved  before  the  next  step  begins. 
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Ticker  -  A  small  Java  Applet  that  displays  a  specific  set 
of  headlines,  information,  etc.  Every  web  page  that 
wants  to  display  a  Ticker  must  add  some  special 
HTML  code  into  the  page.  This  code  ensures  that  the 
JAVA  Applet  is  loaded  from  a  server.  Some  param¬ 
eters  control  the  visible  output  like  coloring  and  of 
course  they  control  which  news  is  loaded.  Visit  http:/ 
/7am. com/ticker/  or  http://www.tickerland.com/. 

Unstructured  Decisions  -  This  type  of  decision 

situation  is  complex,  and  no  standard  solutions  exist 
for  resolving  the  situation.  Some  or  all  of  the 
structural  elements  of  the  decision  situation  are 
undefined,  ill-defined,  or  unknown.  For  example, 
goals  may  be  poorly  defined,  alternatives  may  be 
incomplete  or  noncomparable,  and  choice  criteria 
may  be  hard  to  measure  or  difficult  to  link  to  goals. 

User-friendly  -  An  evaluative  term  for  a  decision 
support  system’s  user  interface.  The  phrase  indicates 
that  users  judge  the  user  interface  as  how  easy  to 
learn,  understand,  and  use  it  is. 

User  Interface  (or  “Human — Computer  Interface”)  - 
The  component  of  a  computerized  support  system 


that  allows  bidirectional  communication  between  the 
system  and  its  user.  This  is  also  called  the  dialogue 
component  of  a  DSS.  An  interface  is  a  set  of 
commands  or  menus  through  which  a  user  commu¬ 
nicates  with  a  program. 

Web-based  DSS  -  A  computerized  system  that  delivers 
decision  support  information  or  decision  support 
tools  to  a  manager  or  business  analyst  using  a  “thin- 
clienf  ’  web  browser  like  Netscape  Navigator  or 
Internet  Explorer.  The  computer  server  that  is 
hosting  the  DSS  application  is  linked  to  the  user’s 
computer  by  a  network  with  the  TCP/IP  protocol.  In 
many  companies,  a  web-based  DSS  is  synonymous 
with  an  enterprise-wide  DSS  that  is  supporting  large 
groups  of  managers  in  a  networked  client-server 
environment  with  a  specialized  data  warehouse  as 
part  of  the  DSS  architecture. 

“What  If’  Analysis  -  The  capability  of  “asking”  the 
software  package  what  the  effect  will  be  of  changing 
some  of  the  input  data  or  independent  variables. 


Glossary 
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Appendix  1.  Background 
Information 

6/Thomas  Gunther,  Office  of  Water  and  Science,  Department  of  the  Interior 


In  1997,  an  Interagency  Group  on  Decision  Support 
(IGDS)  was  formed  to  provide  more  communication 
and  cooperation  among  developers  of  decision  support 
system  (DSS)  tools  and  services  aimed  at  land,  natural 
resources,  and  environmental  management.  The  mission 
of  the  IGDS  is  to  encourage  the  use  of  and  to  advance 
the  state  of  the  art  of  these  applications  of  DSS  tools 
and  systems.  Its  goals  are  to: 

•  Better  achieve  effective  decision  support  capabili¬ 
ties  for  ecosystem  management  through  collabora¬ 
tion. 

•  Establish  a  process  for  working  toward  a  “next- 
generation”  decision  support  capability. 

•  Develop,  distribute,  and  regularly  update  under¬ 
standing  of  existing  decision  support  capabilities. 

•  Identify  manager  and  stakeholder  needs  not 
currently  met. 

•  Facilitate  the  integration  of  evolving  scientific 
understanding  into  decision  support. 

•  Identify  emerging  technologies  and  describe  how 
they  may  contribute  to  decision  support. 

•  Provide  a  forum  for  the  interaction  of  users, 
designers,  and  contributors  to  the  long-term 
development  of  decision  support. 

The  charter  of  the  IGDS  (Lessard  and  Gunther,  1998) 
sets  the  goal  of  improving  the  management  and  protec¬ 
tion  of  natural  resources  and  the  environment  through 
development  and  application  of  decision  support 
systems  that  facilitate  the  decision-making  process  and 
provide  a  suite  of  integrated  tools  and  services.  Two 
major  objectives  are  to  improve  the  interoperability, 
modularity,  and  transferability  of  decision  support  tools 
and  services  and  to  apply  decision  science  principles  to 
natural  resource  and  environmental  decision  making. 

In  a  series  of  meetings  held  in  1997  and  early  1998, 
the  IGDS — with  participants  from  multiple  agencies 
and  the  academic  and  private  sectors — shared  knowl¬ 
edge  of  existing  decision  support  systems  and  applica¬ 
tions  and  discussed  unmet  needs  and  opportunities.  Our 
report  supports  their  five  conclusions  (Lessard  and 
Gunther,  1998): 


•  Powerful  tools  and  systems  are  available  or  are 
under  development. 

•  No  one  system  provides  or  is  likely  to  provide  the 
broad  range  of  capabilities  needed  by  all  decision 
makers  and  stakeholders. 

•  Some  aspects  of  decision  support  are  very  sophisti¬ 
cated,  while  others  are  still  in  their  infancy. 

•  No  one  agency  or  developer  is  likely  to  have  the 
budgetary  resources  or  mission  to  develop  and 
maintain  the  full  range  of  tools  needed  by  decision 
makers. 

•  Evolving  technology,  inadequate  budgets,  and 
increasingly  complex  challenges  of  resource 
management  combine  to  provide  an  opportunity 
for  expanded  cooperation  in  the  development  of 
the  next  generation  of  decision  support. 

The  U.S.  Geological  Survey  (USGS)  formed  a  DSS 
Special  Interest  Group  to  pursue  IGDS  goals  at  the 
bureau  level.  In  October  1998,  the  USGS  Office  of 
Biological  Informatics  and  Outreach  hosted  a  DSS 
workshop  in  Denver,  Colorado  (Getter  and  others, 

1999).  Although  the  USGS  Biological  Resources 
Division  (BRD)  has  considerable  DSS  expertise  and 
capabilities,  no  formal  effort  had  been  made  to  describe 
and  document  these  activities  and  capabilities  prior  to 
this  workshop. 

The  workshop  brought  together  representatives  of 
BRD  science  and  technology  programs  to  initiate  an 
assessment  of  the  division’s  capabilities  and  to  define 
an  appropriate  role  for  BRD  interaction  and  involve¬ 
ment  with  its  partners.  Action  items  developed  from  the 
workshop  were  to  (1)  develop  a  DSS  framework 
document  for  use  in  biological  research,  (2)  develop  a 
“proof  of  concept”  DSS  based  upon  the  framework 
document,  and  (3)  more  fully  integrate  DSS  into  BRD 
program  elements.  This  report,  “A  Framework  for 
Ecological  Decision  Support  Systems:  Building  the 
Right  Systems  and  Building  the  Systems  Right,” 
represents  completion  of  the  first  objective  identified  at 
the  workshop.  Additional  information  concerning  the 
DSS  workshop  can  be  found  at  the  BRD  DSS  Web  site 
(at  http://biology.usgs.gov/dss/),  which  also  houses 
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responses  to  an  interactive  questionnaire  initially  posted 
to  determine  the  USGS  DSS  inventory  and  user 
interests. 

This  report  (1)  presents  a  definition  of  a  DSS  that  is 
consistent  with  that  of  the  IGDS,  (2)  provides  the 
characteristics  and  functionality  of  DSS,  and  (3) 
suggests  generic  steps  for  research  and  development  of 
DSS.  It  stresses  the  importance  of  a  team  approach  in 
developing  a  DSS,  with  early  interaction  among  upper 
management,  users,  and  system  developers,  and  the 


development  of  user-friendly  interfaces  to  ensure  that 
the  DSS  will  be  effectively  used  for  its  intended 
purpose.  By  implementing  the  recommended  proce¬ 
dures  for  verification  and  testing,  users  of  a  DSS  can  be 
assured  that  it  performs  its  intended  functions  and 
measures  up  to  required  standards  of  quality  and 
reliability.  This  biological  framework  for  DSS  develop¬ 
ment  and  use  will  help  the  biological  community  work 
together  in  achieving  mutual  goals. 


Appendix  1  25 


Appendix  2:  Decision  Support 
Capabilities  for  Future  Technology 
Requirements 


Executive  summary 


Edited  by  Gene  Lessard,  Senior  Ecologist,  National  Watershed  Coalition,  Lessard@erols.com 

In  July  2000,  a  decision  support  system  (DSS)  workshop  was  held  in  Denver,  Colorado,  and  jointly  sponsored 
by  the  Interagency  Group  on  Decision  Support  and  the  Aurora  Partnership.  Workshop  discussion  leaders  pro¬ 
vided  the  following  synopses  (printed  verbatim  from  a  workshop  document)  of  their  discussion  groups  to  show 
the  ongoing  dialogue  about  future  DSS  technology  requirements. 

Figures  had  to  be  redrawn  for  printing  purposes  and  are  not  the  authors’  original  versions. 


Historical  perspective 


The  Interagency  Group  on  Decision  Support  (IGDS) 
was  formed  in  1997  to  discuss  decision  support  tools, 
services,  and  other  issues  as  they  relate  to  natural 
resources  and  the  environment.  It  has  provided  a  forum 
for  agency  staff  and  stakeholders  to  meet  and  exchange 
information  on  currently  available  tools,  address 
concepts  and  principles  of  decision  science,  and  to 
explore  the  potential  for  greater  cooperation  and 
coordination  among  users  and  developers.  The  vision  of 
the  Group  is  to  establish  “a  cooperative  group  of  public 
and  private  stakeholders  working  together  to  build 
advanced  systems  for  natural  resource  and  environmen¬ 
tal  decision  making.”  This  vision  led  to  the  establish¬ 
ment  of  the  Aurora  Partnership. 

The  Aurora  Partnership  (AP)  has  a  much  greater 
reach  than  the  IGDS.  The  Partnership  is  a  collaboration 
of  government  and  nongovernment,  university,  and 
private  sector  organizations.  The  approach  of  the 
Partnership  is  to  engage  in  an  open  collaborative 
process  to  share  ideas,  information,  and  technologies  to 
advance  tools  and  systems  that  will  enable  the  practical 
use  of  natural  and  social  science  in  decision  making. 
Their  goal  is  to  “stimulate  the  development  and  applica¬ 
tion  of  the  next  generation  of  information  tools  and 
systems  for  place-based  management  decision  making 
through  the  collaboration  of  public  and  private  stake¬ 
holders.”  Specific  objectives  are  to: 


•  Improve  the  interoperability,  modularity  and 
transferability  of  decision  support  tools  and 
services; 

•  Develop  and  apply  decision  science  principles  to 
place-based  management  decision-  making;  and 

•  Incorporate  both  the  decision  science  principles 
and  tools  into  a  science-based  decision  support 
framework. 


Placed-based  decision-making 


The  process  for  making  decisions  for  “places”  will 
become  increasingly  complex  and  collaborative  as  land 
use  intensifies,  land  values  increase,  more  regulations 
are  formulated,  and  government  authorities  and  non¬ 
government  stakeholder  groups  multiply.  Multiple 
authorities  (such  as  drainage  and  watershed  authorities, 
county  boards,  state  and  federal  regulators,  regional  air 
quality  managers)  will  need  to  negotiate  with  each  other 
and  engage  stakeholders  in  these  negotiations  before 
they  select  a  course  of  action  (or  inaction). 

There  will  be  increasing  demands  for  technology  to 
help  scope  the  issues,  enhance  communications  in 
multiple  media,  identify  and  present  relevant  data, 
manage  and  refine  processes,  picture  future  options  and 
outcomes,  and  channel  participants  towards  endpoints 
and  decisions.  New  technological  capabilities  will 
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emerge  from  academic  research,  industry  investments 
and  government  experiments.  The  question  posed  by 
the  Aurora  Partnership  is  simply  this — can  a  conscious 
effort  by  an  affiliation  of  academics,  industry  and 
government  help  us  more  rapidly  and  effectively  shape 
and  realize  the  potential  advantages  of  these  technolo¬ 
gies? 

Assuming  the  answer  to  this  question  is  yes,  how  can 
we  help  quicken  improvements  in  placed-based 
decisions  through  collaborative  technology  initiatives, 
and  what  are  the  critical  steps?  The  first  step  is  to 
envision  and  communicate  how  we  want  technology  to 
aid  in  our  future  decision  processes.  This  requires  a 
strong  dialogue  between  the  developers/pioneers  of 
different  technology  innovations,  and  placed-based 
decision-  makers.  The  better  we  can  describe  desirable 
end  states  in  decision  processes,  the  better  we  will  be 
able  to  shape  technologies  to  help  us  achieve  these  end 
states.  Conferences,  publications,  talks  and  web  site 
demonstrations  can  all  help  elicit  descriptions  of  these 
future  decision  capabilities  and  nurture  a  strong  and 
interacting  community  of  technology  developers  and 
pioneer  users. 


The  workshop 


This  workshop  is  one  of  a  series  of  workshops  aimed  at 
addressing  the  goals  and  objectives  of  the  IGDS  and  the 
Aurora  Partnership  (For  more  information  see  “Strate¬ 
gic  Plan”  www.aurorapartnership.com)  and  beginning 
the  envisioning  and  communicating  process.  The 
organizing  committee  for  this  workshop  includes  Tom 
Gunther  (USDI),  Bill  Goran  (DoD),  Tom  Hart  (DoD), 
Ken  Snyder  (DoE),  Gary  Fisher  (USDI),  and  Maury 
Nyquist  (USDI). 

Decision  support  systems  have  been  defined  in  many 
ways,  reflecting  different  emphasis  or  points  of  view. 
For  the  purposes  of  this  workshop  we  define  Place- 
based  Decision  Support  Systems  (PBDSS)  as  combina¬ 
tions  of  computer  hardware,  software,  data  and  models 
that  allow  users  to  better  understand  complex  issues, 
develop  alternative  approaches  to  resolving  those 
issues,  and  test  them  within  a  “what  if’  environment. 
Geographic  information  systems  (GIS)  are  clearly  a 
crucial  element  of  such  systems,  but  need  to  be  aug¬ 
mented  by  additional  decision  support  tools. 

The  combination  of  new  capabilities  and  growing 
demand  are  generating  greater  expectations  for  decision 
support,  and  identifying  new  challenges  for  developers 
and  users  alike.  This  workshop  is  intended  to  discuss 
challenges  and  opportunities  to  meet  these  expectations 


and  to  outline  the  next  steps  in  a  strategy  for  the  next 
stage  of  PBDSS  development.  In  discussions,  meetings 
and  workshops  over  the  last  several  years,  the  IGDS  and 
the  Aurora  Partnership  have  identified  several  topical 
areas  that  we  addressed  in  this  workshop: 

•  Decision  processes; 

•  Collaborative  tools  for  decision  support; 

•  Information  management  and  interoperability; 

•  Knowledge  management  and  decision  support; 
and, 

•  Computation,  communication,  and  data  storage. 

Issues  are  explored  within  the  context  of  a  decision¬ 
making  processes  currently  being  used  (Tom  Gunther, 
this  report).  Tom  notes  that  decision-making  is  the 
result  of  a  sequence  of  implicit  or  explicit  steps.  He 
describes  a  generalized  process  for  decision-making, 
which  outlines  the  steps:  process  mapping,  problem 
framing,  intelligence  gathering,  evaluating  and  choosing 
alternatives,  and  learning  from  the  outcome  of  the 
decision.  This  decision  process  is  often  a  function  of  the 
type  of  decision  to  be  made.  For  instance,  decisions 
made  under  the  National  Environmental  Policy  Act 
have  legal,  well-documented,  procedures  to  follow. 
Other  routine  decisions  may  involve  only  a  portion  of 
the  process.  And,  lastly,  collaboration  is  often  an 
essential  ingredient  in  decision-making.  In  summary, 
Tom  notes  that  these  three  aspects — the  process,  the 
type  of  decision,  and  the  need  for  collaboration — 
encompass  the  environment  in  which  decision  support 
systems  and  tools  will  be  used. 

Tom  sees  a  future  in  which  decision-makers  have  “a 
toolbox  containing  a  variety  of  tools  to  help  address  the 
range  of  place-based  problems.  Some  tools  would  assist 
in  understanding  and  improving  the  process  itself; 
others  would  assist  specific  stages,  such  as  intelligence 
gathering  or  evaluating  alternatives;  and  others  would 
help  identify  very  specific  tools  for  specific  problems  or 
parts  of  problems  (e.g.  hydrologic  models).  New  tools 
could  be  added  as  new  problems  arise  or  as  conditions 
change,  without  the  need  to  extensively  modify  or 
replace  either  the  toolbox  or  existing  tools.  The  inter¬ 
face  would  be  consistent,  and  learning  about  the 
capabilities  of  the  toolbox  and  the  geographic  area 
being  considered  would  be  cumulative.  The  toolbox 
would  be  one  of  many  offered  by  different  vendors,  and 
all  would  be  capable  of  incorporating  tools  from  other 
sources.” 

With  collaboration  a  major  part  of  the  environment 
of  decision-making,  it  seems  reasonable  to  develop 
tools  specifically  to  enhance  the  collaborative  process. 
Mike  Case  (this  report)  notes  that  “collaborative  tools 
can  help  in  two  important  ways.  First,  they  can  serve  as 
tools  to  apply  proven  group  decision  support  methods 
and  processes.  Second,  they  can  serve  as  an  aid  in 
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helping  group  members  arrive  at  a  shared  understanding 
of  technical  information  relevant  to  their  issues.”  In 
addition,  collaborative  tools  can  allow  stakeholders  to 
work  at  their  own  time  (asynchronous)  and  place 
(distributed)  versus  the  traditional  “town  meeting” 
venue  (synchronous  and  collocated).  Mike  notes  that 
“effective  collaborative  tools  should  help  stakeholders 
in  a  decision  process  to  arrive  at  better  decisions  when 
measured  against  metrics  of  consensus,  awareness, 
representation,  time,  and/or  cost.” 

To  arrive  at  a  vision  for  the  future  of  collaborative 
tools,  Mike  first  examines  some  of  the  impediments  to 
effective  collaboration  such  as  organizational  competi¬ 
tion,  identification  of  stakeholders,  the  knowledge  and 
skill  of  individual  stakeholders,  and  others.  With  these 
impediments  in  mind,  collaborative  tools  can  accom¬ 
plish  three  goals:  to  develop  shared  purpose,  to  develop 
shared  understanding,  and  to  provide  a  focus  for 
resources. 

Any  decision-making  process  requires  data  that  can 
be  converted  into  information  and,  ultimately,  into 
knowledge.  This  conversion  process  needs  to  be 
managed  in  a  transparent  manner  to  the  user.  Jeff 
Holland  (this  report)  notes  that  “information  manage¬ 
ment  involves  the  purposeful,  directed  manipulation  of 
data  as  it  moves  from  information  (e.g.,  inputs  for 
decision-making)  to  knowledge  (e.g.,  belief  and  value 
sets  used  as  overarching  constraints,  goals,  and  objec¬ 
tives  in  decision-making).”  And,  “interoperability  is 
defined  as  the  capability  of  productively  and  seamlessly 
moving  from  data  to  information  to  knowledge,  and 
back,  in  a  manner  whose  infrastructure  is  transparent.” 

Jeff’s  vision  for  the  future  focuses  around  the 
“seamless  movement  of  data,  information,  and  manage¬ 
ment  from  research  to  analysis  to  synthesis  to  adapta¬ 
tion.  This  is  particularly  true...  where  new  science 
findings  must  flow  to  implementation,  and  where 
feedback  produces  the  need  for  adaptation  of  a  given 
management  decision.”  With  this  in  mind,  Jeff  describes 
five  areas  of  functionality  for  interoperable  information 
systems: 

•  Access:  Differing  stakeholders  utilize  different 
components  of  an  interoperable  information 
system  from  their  geographically  distributed 
desktops. 

•  Catalog:  Provide  for  a  repository  for  techniques/ 
algorithms  in  a  standard,  web-searchable  para¬ 
digm. 

•  Smart  agent/knowledge  management:  Methods  to 
capture  the  knowledge  base,  sharing  “lessons 
learned,”  “decision  model,”  “case  studies,”  etc.  for 
repositing  in  the  cataloging  framework  above. 

•  Motivation:  Develop  mechanisms  that  strongly 
encourage  the  development  of  in-house  and 


marketplace  interoperable  solutions. 

•  Processing:  Interoperability  and  reuse  of  tools 
through  standards. 

Knowledge  is  an  asset  that  needs  to  be  managed. 
However,  knowledge  management  presents  many 
additional  dilemmas.  Wayne  Schmidt  (this  report) 
identifies  knowledge  management  as  “an  integrated, 
systematic  approach  to  identifying,  managing,  and 
sharing  an  enterprise’s  information  assets,  including 
documents,  databases,  policies,  procedures,  and  implicit 
expertise.  The  purpose  of  this  integration  is  to  make 
available  the  validated  ‘Decision  Support  Process’  that 
will  guide  the  user  to  the  decision  point.  Fundamentally, 
knowledge  management  makes  the  collective  informa¬ 
tion  and  experience  of  an  enterprise  available  to  the 
individual  knowledge  worker,  who  is  responsible  for 
using  it  wisely  and  for  replenishing  the  knowledge 
asset.  This  ongoing  cycle  promotes  a  learning  organiza¬ 
tion,  stimulates  collaboration  and  empowers  people  to 
continually  enhance  the  way  they  perform  work.” 

Wayne’s  vision  of  the  future  for  knowledge  manage¬ 
ment  includes  a  “searchable  distributed  library  of 
Decision  Support  Processes”  (DSPs)  with  each  DSP 
being  a  “complete  ‘case  study’  of  the  process,  data 
requirements,  analytic  tools...” 

The  identification  of  a  decision  process,  the  need  for 
and  use  of  collaboration  (Tom  Gunther),  development 
of  collaborative  tools  (Mike  Case),  managing  data, 
information  (Jeff  Holland),  and  Knowledge  (Wayne 
Schmidt)  all  seem  straight  forward.  In  fact,  it  can  be 
argued  that  organizations  are  already  doing  all  of  the 
above.  However,  as  Doug  Johnston  (this  report)  notes 
“the  information  environment  remains  disorganized 
with  relevant  data  distributed  across  many  databases 
and  systems  with  differing  descriptions  based  on 
disciplinary  focus.  Models  for  analysis  or  creation  of 
data  are  similarly  distributed  and  disciplinary.  Thus 
decision  makers,  confronted  with  a  problem  to  solve, 
must  seek  out  relevant  data,  acquire  that  data,  transform 
it  to  a  common  framework,  identify  the  models  that  will 
address  the  problem  being  solved,  acquire  the  computa¬ 
tional  resources  required  to  run  the  models,  and  finally, 
if  all  goes  well,  actually  begin  to  address  the  real 
problem  they  set  out  to  solve.” 

Doug  notes  that  the  objective  of  applying  technology 
to  decision-making  is  to  “make  available  resources 
(computational,  data,  models)  at  the  right  time  to  the 
right  problem.”  The  problem  with  technology  applica¬ 
tion  is  more  institutional  in  nature  than  technical. 
Organizations  choose  not  to  adopt  standards,  use 
protocols,  or  provide  access  to  information.  Doug 
presents  the  concept  of  a  “broker”  which  represents  a 
“tiered  set  of  services  built  to  connect  the  technologies 
in  ways  that  can  bring  existing  capabilities  to  bear  fruit 
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quickly.”  A  broker  filters  through  available  options  and 
presents  a  limited  set  of  comparable  options.  The 
foundation  of  the  broker  concept  is  the  catalog.  The 
catalog  contains  available  elements  or  products  and  a 
method  for  querying.  An  abstraction  defines  attributes, 
a  definition  translator  matches  abstract  definitions  to 
actual  elements,  and  a  store  provides  a  set  of  pointers  to 
the  actual  resources. 


Next  steps  in  the  strategy 


A  number  of  themes  were  emphasized  by  each  of  the 
work  groups.  The  first  of  these  was  the  need  for 
demonstration  programs  focused  only  in  part  on 
addressing  a  specific  local  issue.  A  clear  additional  goal 
of  these  demonstration  programs  would  be  to  leave  a 
legacy  of  capability  within  the  community  that  will  be 
applied  to  future  issues. 

Place-based  issues  are  seldom  one  time  one  decision 
in  nature — rather,  the  issues  need  to  be  continuously 
addressed  and  the  public  and  stakeholders  frequently 
informed.  One  important  goal  of  a  program  to  develop 
and  demonstrate  improved  place-based  decision-making 
is  to  grow  sustainable  capabilities  for  localities  and 
regions.  To  help  encourage  this  capacity  building, 
demonstration  and  test-bed  programs  should  use  one 
metric  for  success  relating  to  the  “sustainable  capability 
creation.”  This  capacity  building  has  several  dimen¬ 
sions — have  stakeholders  learned  to  work  better 
together?  Is  there  local  available  expertise  in  the 
technologies  needed  to  support  improved  place-based 
decisions?  Is  there  a  process  in  place  to  obtain  relevant 
data — especially  dynamic  data  elements?  Is  there  an 
improved  climate  in  the  community/region  for  decision¬ 
making? 

Demonstrations  can  be  conducted  within  existing 
federal,  state  or  privately  sponsored  programs,  or  could 
be  a  context  for  fashioning  new  programs.  Metrics  for 
success  would  not  just  be  related  to  the  successful  use 
of  some  new  capability  in  a  decision  process,  but  the 
successful  transition  of  this  capability  into  the  local 
community,  and  the  contribution  of  lessons  learned 
about  this  capability  back  to  a  network  of  developers 
and  other  potential  placed-based  users.  Metadata  about 
applications  and  lessons  learned  would  be  contributed 
towards  web  accessible  repositories  to  help  inform 
increase  the  body  of  available  expertise  and  experts. 

Specific  recommendations  for  demonstration 
programs  from  the  workshop  are: 

•  Develop  a  set  of  case  studies  and/or  other  “lessons 
learned,”  including  evaluations  of  tools  and 


applications,  meta-analyses  and  retrospectives  on 
past  place-based  activities. 

•  Initiate  a  series  of  linked  demonstration  projects, 
each  of  which  will  be  designed  to  improve  the 
support  for  the  decision  process. 

•  Develop  and  conduct  a  series  of  highly-focused 
technology  that,  by  design,  investigate  the  better  of 
several  means  for  achieving  interoperable  informa¬ 
tion  technologies.  Envisioned  is  a  set  of  five  to 
seven  demonstrations,  chosen  to  reflect  differing 
natural  and  water  resources  decision  processes, 
each  with  differing  stakeholder  requirements,  that 
exercises  the  differing  interoperability  and  infor¬ 
mation  aspects  listed  above.  It  is  recommended 
that  the  Federal  government  agencies  with  major 
roles  in  the  natural  and  water  resources  area  come 
together  to  establish  these  demonstrations  in  direct 
concert  with  stakeholder  groups  and  industry. 

•  Develop  a  case  study.  Build  a  set  of  DSPs  for  a 
particular  problem  domain. 

•  Integrate  a  lessons  learned  capability  and  a  way  for 
users  to  evaluate  the  DSPs. 

•  Focus  on  the  establishment  of  a  national  center  or 
centers  that  will  coordinate  a  small  number  of  test¬ 
bed  programs  while  providing  collaborative 
implementation  toolkits  to  groups  that  are  inter¬ 
ested  in  experimenting  with  place-based  decision 
support.  An  alliance  of  federal  and  state  agencies, 
standards  organizations,  vendors,  and  consultants 
should  pool  resources  on  a  voluntary  basis  to: 

♦  Encourage  local  centers  for  Place-based  Deci¬ 
sion  Support; 

♦  Act  as  a  repository  and  resource  for  “best 
practice”  process  templates; 

♦  Act  as  a  repository  and  resource  for  lessons 
learned; 

♦  Provide  free  or  low  cost  visualization,  modeling, 
and  scenario  generation  tools;  and, 

♦  Fund  a  small  number  of  pilot  demonstration 
programs. 

Creation  of  repositories  and  catalogue  services  is 

another  important  step  in  facilitating  local  decision 
making.  Such  services  are  needed  to  ensure  that 
relevant  data,  expertise,  software  and  process  informa¬ 
tion  all  gets  to  the  right  persons  at  the  right  time  in  the 
right  format.  Creation  of  networks  of  services  that  are 
“registered”  against  local  applications  will  help  connect 
relevant  experts,  data  and  expertise  with  local  commu¬ 
nity  and  regional  efforts.  Applications  of  decision 
technology  both  draw  from  and  contribute  towards 
these  repositories  and  catalogue  services — and  the 
network  of  services  grows  in  terms  of  both  the  avail¬ 
ability  of  data  and  the  processing  of  relevant  informa¬ 
tion  with  each  application  conducted. 
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Specific  recommendations  for  creation  of  reposito¬ 
ries  and  catalogue  services  from  the  workshop  are: 

•  Develop  open  “wizards”  that  facilitate  placing  data 
within  web-accessible  repositories.  Define  these 
repositories  using  standards  as  well  so  that  they 
can  be  queried  and  mined  by  different  users  who 
see  them  as  virtual  repositories. 

•  Develop  a  consistent,  integrated  problem  solving 
computational  environment,  a  “decisioning”  world 
on  the  web,  is  needed.  Major  Federal  government 
organizations  and  industry  should  collaborate  to 
provide  the  critical  mass  needed  to  properly 
develop  the  environment. 

•  Develop  a  standard  set  of  agents,  acting  behind  the 
scenes  over  a  network  or  the  web,  to  facilitate  data/ 
information  creation,  location,  and  retrieval, 
model/tool  selection,  visualization,  etc. 

•  Develop  a  search  and  presentation  scheme  for 
users. 

•  Develop  a  web  site  to  improve  user  access. 

Another  important  step  is  to  develop  and  implement 

standards  for  interoperability  of  capabilities — so  that 
new  functions  easily  fit  into  an  extensible  framework, 
and  all  capabilities  from  any  vendor  or  data  store  can  be 
used  together.  The  Open  GIS  Consortium  is  dedicated 
to  this  end  state,  and  various  other  standards  groups  are 
focusing  on  components  of  this  goal.  Such  efforts  are 
part  of  the  critical  path  for  future  decision-making, 
because  of  the  need  to  share  and  access  data  and  tools 
and  expertise  relevant  to  any  local  issue  and  the  need  to 
share  across  localities  and  regions. 

Specific  recommendations  for  standards  for 
interoperability  from  the  workshop  are: 

•  Develop  and  utilize  one  set  of  national 
interoperability  standards. 

•  Provide  financial  incentives  through  contracts  to 
promote  the  use  of  established  standards. 

•  Eliminate  designed  non-interoperability  (“not 
invented  here”) — promote  the  use  of  standards 
through  Agency  order  at  the  national  or  agency 
level. 

•  Make  new  technology  implementations  transparent 
to  the  user — this  would  be  done  primarily  through 
the  use  of  object-oriented,  modular  developments 
and  the  use  of  standards/agents  as  listed  above. 

Besides  implementing  standards,  agencies  need  to 
develop  a  framework  approach  into  which  all  of  their 
data,  models  and  commercial  capabilities  are  mapped. 
These  frameworks  are  needed  to  get  disparate  technol¬ 
ogy  pieces  (technical  systems,  business  systems, 
commercial  software,  data  bases)  into  a  self-improving, 
scalable  and  evolving  and  web-based  context.  This 
“framework”  needs  to  eliminate  requirements  for 
duplicating  data  entry  across  systems;  automatically 


gathers  data  on  users,  applications,  processes  and  usage 
context;  create  templates  and  process  flows  to  reduce 
steps  and  avoid  costs  for  future  efforts;  and  draw  upon 
agency  wide  stores  of  data  and  tools  to  seek  experts  and 
experience  relevant  to  any  specific  task.  Agency 
frameworks  can  drive  the  “interoperability”  context  for 
industry  products,  and  can  be  mimicked  at  regional  and 
local  levels. 

Specific  recommendations  for  developing  a 
framework  approach  from  the  workshop  are: 

•  Develop  and  inventory  tools  that  can  help  assess 
decision  processes  (rather  than  decisions). 

•  Develop  a  set  of  guides  (probably  through  a  World 
Wide  Web  site)  to  the  process  and  the  currently 
available  tools  and  systems.  Identify  specific  tools 
for  different  stages  of  the  decision  process. 

•  Establish  a  new  set  of  smart  agents  or  wizards  to 
monitor  and  feedback  the  actual  processes  used  in 
decision  making  for  a  given  situation. 

•  Provide  a  higher  conceptual  or  natural  language 
query  capability  to  facilitate  decision  support  to 
many  stakeholders — develop  the  means  for 
decision-makers  to  query  data  of  all  types  (includ¬ 
ing  modeling  and  simulation  data)  using  terms 
(language)  they  understand  and  identify  with. 

•  Define  a  model  for  a  Decision  Support  Process 
(DSP).  This  would  include  attributes  and  methods 
for  combining  DSPs  and  updating  them  as  tools 
change. 

•  Address  the  issue  of  getting  new  DSPs.  Perhaps 
there  could  be  contractual  language  that  would 
require  contractors  to  input  their  process  and 
results  in  an  electronic  form  in  addition  to  the 
normal  printed  documents. 

•  Inventory  computerized  tools  that  are  currently 
available  and  which,  with  some  modification,  can 
aid  in  decision-making. 

•  Initiate  a  “Request  for  Information”  for  PBDSS 
tools  and  systems,  and  begin  a  multi-agency 
evaluation  and  analysis  of  these  tools  and  systems. 


Who  has  what  role? 


There  are  roles  for  government  organizations  (at 
multiple  tiers),  educational  institutions,  industry,  and 
non-government  organizations  aimed  at  shaping 
improved  future  placed-based  decision  making. 
However,  the  roles  for  any  one  task  are  not  necessarily 
distinct.  For  instance,  research  can  be  performed  by 
government,  academics,  and  private  firms  alike.  In 
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general,  the  roles  for  the  various  players  are  outlined 
below: 

•  Communicate  a  Vision 

♦  Communities  and  Regions  -  Help  to  shape  vision 

♦  Federal  Agencies  and  Organizations  -  Participate 
in  shaping  vision/running  workshops 

♦  Consortia  and  Standards  Organizations  -  Build 
standards  responsive  to  vision/  approach 

♦  Academics  -  Conduct  research  and  education 
towards  future  decision  technologies  and 
processes 

♦  Private  Firms  -  Affirm  vision  with  responsive 
products  and  services 

•  Demonstrations 

♦  Communities  and  Regions  -  Conduct  demos  on 
local  issues 

♦  Federal  Agencies  and  Organizations  -  Provide 
resources  for  demo  programs 

♦  Consortia  and  Standards  Organizations  -  Operate 
test-beds 

4  Academics  -  Participate/assist  in  demos 

4  Private  Firms  -  Provide  resource  assistance 

•  Repositories  and  Templates 

4  Communities  and  Regions  -  Draw  from  and 
contribute  to  repositories 

4  Federal  Agencies  and  Organizations  -  Build 
repositories  for  objects,  lessons  learned,  pro¬ 
cesses 

4  Consortia  and  Standards  Organizations  -  Help 
nurture  standard  approaches  for  repositories 

4  Academics  -  Develop  concepts  and  prototypes 

4  Private  Firms  -  Develop  capabilities  that  utilize 
repositories 


•  Interoperative  Products 

4  Communities  and  Regions  -  Use/require 
interoperable  products 

4  Federal  Agencies  and  Organizations  -  Use/ 
require  interoperable  products  and  develop 
scopes  of  work 

4  Consortia  and  Standards  Organizations  -  Focus 
standards  efforts  on  inter-operability 
4  Academics  -  Develop  concepts  and  participate  in 
test-beds 

4  Private  Firms  -  Form  consortia  and  provide  test¬ 
bed  involvement  to  make  interoperable  tools 

•  Build  Local  Capacity/Experts 

4  Communities  and  Regions  -  Build  sustaining 
capability  through  demos 

4  Federal  Agencies  and  Organizations  -  Encourage 
capability  building  through  demo  programs 
4  Consortia  and  Standards  Organizations  -  Use 
test-beds  to  help  build/sustain  capacity 
4  Academics  -  Collaborate  with  local/regional 
entities  to  build/sustain  capabilities 
4  Private  Firms  -  Focus  grants  and  assistance 
towards  capability  building 

•  Building  Frameworks 

4  Communities  and  Regions  -  Focus  market 
demands  towards  highest  needs 
4  Federal  Agencies  and  Organizations  -  Create 
overarching  information  technology  concepts 
and  approaches  for  “seamless”  capabilities 
4  Consortia  and  Standards  Organizations  - 
Develop  standards  that  facilitate  frameworks 
4  Academics  -  Develop  concepts  and  prototypes 
4  Private  Firms  -  Build  tools  and  services  that  fit 
into,  expand  and  are  compatible  across  frame¬ 
works 
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Appendix  2:  Continued 


Decision  Processes 

By  Thomas  Gunther,  Office  of  Water  and  Science,  U.S.  Department  of  the  Interior,  Washington,  DC  20420 


Introduction 


Decision-making  is  the  result  of  a  sequence  of  implicit 
or  explicit  steps,  represented,  in  one  fashion,  in  figure  L 
It  begins  with  the  recognition  of  a  problem  or  an 
opportunity,  or  just  a  vague  awareness  that  some 
condition  or  thing  could  be  improved.  In  this  represen¬ 
tation,  process  mapping,  problem  framing,  intelligence 
gathering,  evaluating  and  choosing  alternatives,  and 
learning  from  the  outcome  follows  recognition  (While 
there  are  many  other  ways  to  represent  the  decision 
process,  they  are,  for  the  purposes  of  this  paper, 
interchangeable).  Of  course,  the  activity  or  experience 
at  any  one  stage — say  intelligence  gathering — can  alter 
the  outcome  of  an  earlier  stage — in  this  case,  perhaps 
problem  framing.  Indeed  iteration  through  such 
“feedback  loops”  is  essential  to  “good”  decision 
making,  allowing  new  information,  knowledge  and 
ideas  to  be  incorporated,  and  permitting  adaptive 
management.  “Good”  decisions  are,  at  least  in  part,  a 
function  of  the  amount  of  time  and  effort  expended  on 
the  process  and  at  each  of  the  stages. 

The  process  of  making  decisions  also  varies  with  the 
type  of  decision  and  the  number  of  decision-makers. 
Some  decisions,  such  as  those  covered  by  the  National 
Environmental  Policy  Act,  must  follow  procedures 
established  by  law  and  be  well  documented.  Some  are 
dependent  on  data  delivered  by  real  time  sensors  or 
experiments,  and  thus  require  rapid  integration  of  data 
and  models  to  meet  management  needs.  Others  are 
routine  management  decisions  regarding  operations  or 
maintenance.  And  some  decisions,  usually  affecting 
longer-term  plans,  require  the  collaboration  of  a  range 
of  stakeholders,  often  with  conflicting  goals  and 
objectives. 

These  three  aspects  of  decision-making— the 
process,  the  type  of  decision,  and  the  amount  of 
collaboration  necessary — constitute  the  environment  in 
which  decision  support  tools  and  systems  will  be  used. 
The  same  set  of  tools  may  provide  support  for  different 
types  and  stages  of  decision-making,  but  be  applied  in 
different  ways,  with  different  intensity,  and  with 
different  requirements  for  precision  and  presentation. 


State-of-the-art  in  decision  processes 


Decision  support  tools  and  systems  cannot  supplant  the 
decision  making  process.  However,  there  are  an 
increasing  number  of  computerized  tools  that  can 
provide  assistance,  and  they  are  becoming  increasingly 
sophisticated.  There  are  tools  to  help  brainstorm, 
discover  and  measure  preferences  and  performance, 
poll  participants,  facilitate  and  manage  meetings,  and 
provide  a  formal  record  of  the  process.  There  are  other 
tools  to  assist  in  understanding  and  displaying  the 
impacts  of  proposals,  including  the  trade-off  among 
various  criteria  with  different  alternatives,  changes  in 
the  way  an  area  will  appear,  and  other  often  non- 
quantifiable  changes  in  the  quality  of  life.  But  many  of 
these  tools  come  from  different  specialties,  disciplines, 
or  developers,  and  they  each  tend  to  focus  on  a  rela¬ 
tively  narrow  issue.  They  are  developed  and  used 
independently  of  each  other,  and  moving  data,  stake¬ 
holder  input,  models  and  results  from  one  to  another  is 
difficult  or  impossible.  Even  if  a  community  is  aware  of 
the  range  of  tools  that  can  assist  them  in  making 
decisions,  the  tasks  of  assembling,  learning,  and 
applying  them  puts  these  capabilities  beyond  the  reach 
of  most  places. 


Vision  for  the  future 


A  manager,  group,  or  community  would  have  a  toolbox 
containing  a  variety  of  tools  to  help  address  the  range  of 
place-based  problems.  Some  tools  would  assist  in 
understanding  and  improving  the  process  itself;  others 
would  assist  specific  stages,  such  as  intelligence 
gathering  or  evaluating  alternatives;  and  others  would 
help  identify  very  specific  tools  for  specific  problems  or 
parts  of  problems  (e.g.,  hydrologic  models).  New  tools 
could  be  added  as  new  problems  arise  or  as  conditions 
change,  without  the  need  to  extensively  modify  or 
replace  either  the  toolbox  or  existing  tools.  The  inter¬ 
face  would  be  consistent,  and  learning  about  the 
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Recognition  -  recognizing  problem  or  opportunity. 
Raising  issue  in  formal  decision  making  context 


Process  Mapping  -  deciding  how  issue  will  be 
resolved,  and  who  will  decide. 


Problem  Framing-  describing  the  problem  to  be 
solved  (or  opportunity  to  be  captured). 


Defining  Goals  and  Criteria  -  selecting  indicators 
and  measures  that  guide  decision  making  in  terms 
of  what  is  sought,  and  how  success  will  be  measured. 


Intelligence  Gathering  -  collecting  and  integrating 
information  that  will  support  problem  framing,  and 
evaluating  the  consequences  of  alternatives. 


Evaluating  and  Choosing  Alternatives  -  comparing 
alternative  courses  of  action  on  multiple  and  often 
competing  criteria. 


Learning  -  using  the  experience  gained  to  refine 
management,  goals,  criteria,  and  the  decision  process. 


Figure  1.  Representation  of  the  decision  process  (adapted  from  Dave  Cleaves  in 
"Report  on  the  Decision  Support  Systems  Workshop,"  USGS,  May,  1999.) 


capabilities  of  the  toolbox  and  the  geographic  area 
being  considered  would  be  cumulative.  The  toolbox 
would  be  one  of  many  offered  by  different  vendors,  and 
all  would  be  capable  of  incorporating  tools  from  other 
sources. 


Strategies  for  implementation 


Several  specific  steps  that  should  be  taken  to  improve 
support  for  the  decision  making  process  are  given 
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below.  They  are  intended  to  increase  user  awareness  of 
existing  tools  (whether  or  not  they  are  designed  for 
place-based  management),  and  move  toward  the 
concept  of  integrated  toolboxes. 

•  Develop  and  inventory  tools  that  can  help  assess 
decision  processes  (rather  than  decisions).  In  most 
cases,  these  tools  will  be  in  the  form  of  paper 
guidelines.  Is  the  range  of  stakeholder  goals  and 
values  identified  and  reflected?  Have  criteria  for 
selecting  among  alternatives  been  established  and 
are  performance  measures  selected? 

•  Inventory  computerized  tools  that  are  currently 
available  and  which,  with  some  modification,  can 
aid  in  decision  making.  Examples  include 
“groupware,”  tools  to  elicit  preferences  and 
weights,  project  management  software,  and 
visualization  programs.  These  tools  were  typically 
developed  for  different  applications  (such  as 


software  development),  but  have  many  capabilities 
important  to  place-based  management. 

•  Initiate  a  “Request  for  Information”  for  PBDSS 
tools  and  systems,  and  begin  a  multi-agency 
evaluation  and  analysis  of  these  tools  and  systems. 

•  Develop  a  set  of  guides  (probably  through  a  World 
Wide  Web  site)  to  the  process  and  the  currently 
available  tools  and  systems.  Identify  specific  tools 
for  different  stages  of  the  decision  process. 

•  Develop  a  set  of  case  studies  and/or  other  “lessons 
learned,”  including  evaluations  of  tools  and 
applications,  meta-analyses  and  retrospectives  on 
past  place-based  activities. 

•  Initiate  a  series  of  linked  demonstration  projects, 
each  of  which  will  be  designed  to  improve  the 
support  for  the  decision  process. 
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Appendix  2:  Continued 


Collaborative  Tools  for  Decision  Support 

By  Mike  Case,  U.S.  Army  Engineer  Research  and  Development  Center,  Champaign,  Illinois  61826-9005 


Introduction 


In  today’s  public  and  private  decision  environments, 
many  choices  formerly  made  by  individuals  are  being 
made  by  groups  of  people.  In  addressing  the  question  of 
how  collaborative  tools  might  help  these  groups,  it  is 
important  to  recognize  that  effective  collaboration  is 
primarily  a  social  process  that  requires  an  understand¬ 
ing  of  human  behavior.  In  the  “place-based”  decision 
arena,  collaboration  tools  can  help  in  two  important 
ways.  First,  they  can  serve  as  tools  to  apply  proven 
group  decision  support  methods  and  processes.  Second, 
they  can  serve  as  an  aid  in  helping  group  members  to 
arrive  at  a  shared  understanding  of  technical  informa¬ 
tion  relevant  to  their  issues. 

Effective  collaborative  tools  should  help  stakeholders 
in  a  decision  process  to  arrive  at  better  decisions  when 
measured  against  metrics  of  consensus,  awareness, 
representation,  time,  and/or  cost.  Collaborative  tools 
may  help  groups  to  develop  shared  understanding  of 
issues,  evaluate  alternative  solutions,  build  consensus, 
or  resolve  disputes.  Tools  may  also  be  used  to  break 
down  barriers  of  time  and  space  in  communication 
between  stakeholders.  In  addition  to  the  traditional 
“town  meeting”  venue  (synchronous  and  collocated), 
tools  allow  collaborators  to  work  at  their  own  time 
(asynchronous)  and  own  place  (distributed).  Although  it 
is  generally  recognized  that  the  bandwidth  of  human 
communication  is  greatest  in  face-to-face  meetings  of 
stakeholders,  the  asynchronous  and  distributed  capabili¬ 
ties  offered  by  automated  collaborative  tools  can  act  to 
augment  limited  resources  of  time  and  funding. 

State-of-the-art  in  collaborative  tools 
and  decision  support 

The  current  status  of  collaboration  tools  encompasses 
both  traditional  and  electronic  media.  Place-based 
decision  support  requires  proactive  efforts  to  meet  with 
stakeholders  and  solicit  involvement.  Techniques  such 


as  focus  groups,  breakout  groups,  flip  charts,  and 
“place-the-dot”  style  voting  are  commonly  used 
techniques.  Despite  their  low  level  of  technology,  they 
may  still  be  considered  state-of-the  art. 

Electronic  media  has  enhanced  rather  than  replaced 
many  of  the  manual  methods  mentioned  above.  Tools 
such  as  e-mail,  mailing  lists,  chat  rooms,  web  servers, 
and  discussion  forums  are  all  examples  of  on-line 
technologies  that  ease  the  process  of  distributing 
information  and  exchanging  points  of  view.  Online 
meeting  tools  such  as  Netmeeting  and  C-U  See  Me 
offer  new  ways  to  overcome  barriers  of  time  and 
distance  through  video  teleconferencing  and  application 
sharing. 

The  class  of  tools  commonly  called  Group  Decision 
Support  Systems  (GDSS)  provides  utilities  to  improve 
elicitation  of  viewpoints,  generation  of  ideas,  discus¬ 
sion,  and  consensus  building.  Typically,  such  systems 
are  set  up  in  a  dedicated  room  in  which  each  participant 
has  their  own  screen  and  keyboard.  Meetings  may  be 
facilitated  or  not  and  input  may  or  may  not  be  anony¬ 
mous. 

A  final  class  of  collaboration  tools  gives  collaborat¬ 
ing  groups  an  online  location  to  keep  shared  documents 
and  data.  These  tools  offer  electronic  document 
management  functions,  communication  utilities  for 
members  (forums,  chat,  etc.),  and  shared  visualization 
of  graphics  (CAD,  GIS,  images).  These  tools  are  readily 
available  over  the  Internet  and  accessible  through  web 
browsers  or  client  software.  Discipline  specific  varieties 
of  these  tools  add  features  needed  to  perform  tasks 
unique  to  a  discipline.  For  instance,  software  configura¬ 
tion  control  tools  offer  utilities  for  version  control  and 
defect  management. 


Vision  for  the  future 


If  the  goal  is  to  improve  the  effectiveness  of  place- 
based  collaboration,  how  can  collaboration  tools  help? 
To  answer  this  question,  let’s  examine  some  impedi¬ 
ments  to  effective  collaboration  and  suggest  some 
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strategies  to  overcome  them.  Having  done  that,  we  will 
suggest  some  ways  in  which  tools  can  help. 

1 .  Competition.  People  and  organizations  (stake¬ 
holders)  have  valid  reasons  to  compete  for  funds, 
space,  resources,  recognition,  or  other  goals. 
Collaboration  is  impeded  by  a  lack  of  common 
goals.  Win-win  strategies  are  desirable,  but  not 
always  possible. 

2.  Turf.  Stakeholders  seek  control  over  decisions 
that  affect  them.  A  real  or  perceived  loss  of 
control  may  impede  collaboration. 

3.  Stovepipes.  Organizational  boundaries  of 
responsibility,  accountability  and  authority  may 
prevent  collaboration. 

4.  Efficiency.  Developing  consensus  among  all 
involved  stakeholders  can  be  difficult  and 
lengthy  process.  Sometimes  it  appears  easier  to 
work  alone  or  in  small  groups.  This  can  be  an 
unfortunate  strategy  when  stakeholders  that  were 
not  considered  put  up  road  blocks  (see  #5). 

5.  Identification  of  Stakeholders.  Sometimes  it  is 
very  difficult  to  recognize  who  the  stakeholders 
are  in  an  issue.  Late  involvement  of  stakeholders 
sets  back  the  collaboration  process. 

6.  Purpose.  Members  of  a  group  may  not  always 
have  a  shared  understanding  to  the  desired 
outcome  of  collaboration. 

7.  Process.  Lack  of  buy-in  to  process  and  tools  can 
inhibit  collaboration. 

8.  Knowledge  and  Skill.  Stakeholders  may  bring 
very  different  backgrounds,  skills,  and  under¬ 
standing  to  a  problem.  Some  may  not  understand 
important  social  or  technical  issues. 

9.  Resistance  to  Change.  Some  stakeholders  may 
be  more  comfortable  with  maintaining  the  status 
quo. 

10.  Resource  limits.  Collaboration  requires  an 
investment  of  resources  (i.e.,  time,  personnel, 
energy,  and  funds). 

Given  these  typical  impediments  to  collaboration, 
how  can  automated  tools  provide  assistance?  In  the 
desired  future,  strategies  and  tools  will  be  available  to 
develop  shared  purpose,  to  develop  shared  understand¬ 
ing,  and  to  provide  resources.  Each  of  these  three  goals 
is  addressed  below. 

•  Shared  Purpose.  Although  stakeholders  may  have 
conflicting  goals,  it  is  important  that  these  goals  be 
articulated  and  expressed.  Only  then  can  a  group 
work  towards  achieving  an  acceptable  solution. 
Resources  will  be  available  that  will  help  groups  to 
achieve  a  preferred  future  focus.  Rather  than 
concentrating  on  overcoming  problems,  proven 
decision  support  strategies  are  available  that  can 
help  groups  to  develop  and  realize  a  shared  vision. 


Group  Decision  Support  (GDSS)  Systems, 
Computer  Supported  Collaborative  Work  (CSCW) 
Tools,  and  lessons-learned  systems  can  assist  in 
overcoming  impediments  1,  2,  3,  4,  5,  6,  and  7. 

•  Shared  Understanding.  Work  in  the  social 
sciences  shows  that  groups  must  work  through  a 
process  of  developing  a  shared  understanding  of 
issues.  This  is  especially  critical  when  technical 
issues  or  jargon  are  involved.  Tools  that  can  help 
illustrate  the  meaning  of  highly  technical  analyses 
to  people  of  disparate  education  and  backgrounds 
will  help  groups  to  achieve  shared  understanding 
more  quickly.  For  example,  visualization  and 
simulation  using  GIS,  3D  and  4D  (time  if  the 
fourth  dimension)  are  very  effective  in  communi¬ 
cating  complex  ideas  and  consequences.  Impedi¬ 
ments  6,  7,  and  8  are  affected. 

•  Provide  Resources.  Traveling  to  meetings, 
arranging  schedules,  or  retrieving  obscure  infor¬ 
mation  all  require  time,  money,  and  personal 
motivation.  Web-based  tools  that  improve  access  to 
materials,  information,  other  stakeholders,  will 
lower  resource-driven  barriers.  Home,  civic,  or 
regionally-oriented  information  resources  can 
make  it  easier  for  local  citizenry  to  research  issues 
on  their  own  time  and  schedule.  One  suggestion  is 
to  set  up  resource  centers  in  libraries  or  local 
schools.  Readily  available  electronic  meeting 
rooms  will  provide  virtual  locations  for  people  to 
meet.  Public  access  sites  to  these  virtual  resources 
will  help  increase  usage.  Impediments  5,  8,  and  10 
are  affected. 


Strategies  for  implementation 


It  is  apparent  that  many  tools  are  already  available  that 
will  help  achieve  the  desired  future.  The  recommended 
strategy  is  to  focus  on  the  establishment  of  a  national 
center  or  centers  that  will  coordinate  a  small  number  of 
test  bed  programs  while  providing  collaborative 
implementation  toolkits  to  groups  that  are  interested  in 
experimenting  with  place-based  decision  support.  An 
alliance  of  federal  and  state  agencies,  standards  organi¬ 
zations,  vendors,  and  consultants  should  pool  resources 
on  a  voluntary  basis  to: 

•  Encourage  local  centers  for  Place-based  Decision 
Support; 

•  Act  as  a  repository  and  resource  for  “best  practice” 
process  templates; 

•  Act  as  a  repository  and  resource  for  lessons 
learned; 
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Provide  free  or  low  cost  visualization,  modeling, 
and  scenario  generation  tools;  and, 

Fund  a  small  number  of  pilot  demonstration 
programs. 


Resources  from  this  alliance  would  be  available  on 
the  condition  that  the  organization  or  individual  using 
the  resources  provide  lessons  learned  or  add  to  the 
knowledge  base  of  process  templates. 
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Appendix  2:  Continued 


Information  Management  and  Interoperability 

By  Jeff  Holland,  U.S.  Army  Engineer  Research  and  Development  Center,  Vicksburg,  MS  39180-6100 


Introduction 


Information  management  involves  the  purposeful, 
directed  manipulation  of  data  as  it  moves  from  informa¬ 
tion  (e.g.,  inputs  for  decision  making)  to  knowledge 
(e.g.,  belief  and  value  sets  used  as  overarching  con¬ 
straints,  goals,  and  objectives  in  decision  making).  Data 
are  formatted,  evaluated,  and  distilled  in  the  process  of 
becoming  information  as  shown  in  the  schematic 
(Figure  2)  below.  Information  is  evaluated,  integrated, 
and  applied  in  the  process  of  becoming  knowledge. 

Interoperability  (“I*”  in  Figure  2)  is  the  capability  of 
productively  and  seamlessly  moving  from  data  to 
information  to  knowledge,  and  back,  in  a  manner  whose 
infrastructure  is  transparent. 

State-of-the-art  in  information 
management  and  interoperability 

The  current  state  of  practice  for  information  manage¬ 
ment  and  interoperability  is  provided  in  bullet  format 
below. 

•  Developing  Standards  Poorly  Used.  There  are 
several  groups  developing  (or  have  developed) 
standards  that  will  strongly  facilitate  interoperable 
information  management.  These  groups  include: 

♦  OGC-Open  GIS  Consortium 


Science 

Community 

User 

Data 

i* 

Information  1*  Knowledge 

formatted 
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results 

integrated 
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Figure  2.  Conceptual  view  of  information  management  and 
interoperability. 


♦  ISO-International  Standards  Organization 

♦  FGDC-Federal  Geodetic  Data  Committee 

♦  Industry-through  the  organizations  above  and 
through  marketplace  activities  related  to  com¬ 
mercial  product  development 

Clearly,  there  are  a  number  of  existing  and  oncoming 
efforts  at  developing  standards.  However,  the  use  of 
these  standards  is  highly  non-standard  between,  and 
within,  major  organizations. 

•  No  Money  to  Apply  Standards.  Organizations  do 
not  yet  appear  to  be  resourcing  the  use  of  stan¬ 
dards  in  their  activities  to  an  adequate  level.  This 
seems  particularly  true  of  efforts  to  reformulate 
existing  software  and  databases  within  new 
standards  contexts. 

Middle-ware  development.  Significant  strides  have 
been  made  in  the  development  of  middle-ware  to 
support  interoperable  information  management. 

•  Proprietary  Solutions.  There  are  a  number  of  in- 
house  proprietary  methods  for  achieving 
interoperability  within  a  given  organization  or 
element  that  do  not  translate  across  the  information 
spectrum. 

•  Emergence/Development  of  Cross  Cutting  For¬ 
mats.  There  are  several  data/information  formats 
that  are  emerging  (e.g.,  XML,  GML,  JAVA,  etc.) 
that  offer  significant  promise  in  supporting 
interoperability.  However,  these  technologies  have 
not  been  fully  implemented. 

Understanding  the  role  of  each  of  these  items 
requires  one  to  have  a  broader  grasp  of  the  current  state 
of  users  of  data,  information,  and  knowledge.  This  state 
is  presented  below. 

•  Generators  vs.  Users.  There  is  a  significant 
difference  in  the  expectations  of,  and  capability 
for,  information  management  between  generators 
of  information  (e.g.,  scientific  and  engineering 
community)  and  the  users  of  information  (e.g., 
decision  makers,  the  public,  etc.).  This  involves  all 
facets  of  information  flow  as  shown  in  Figure  2. 

•  Internet  has  Raised  the  Level  of  Expectations. 

Users  from  all  levels  of  sophistication  now  believe 
that  data,  information,  and  knowledge  should  be 
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available  at  their  finger  tips  through  the  World- 
Wide  Web.  This  raises  the  level  of  expectation 
regarding  the  productive  use  of  the  Internet  in 
reaching  information. 

Institutional  Barriers.  Institutions  often  have  a 
“not-invented-here”  philosophy  that  acts  as  an 
impediment  to  the  use  of  standards  that  affects 
interoperability. 

Lack  of  Reward.  There  are  limited  financial 
incentives  for  industry  to  develop  interoperable 
marketplace  solutions.  Further,  there  is  a  percep¬ 
tion  within  government  organizations  that 
interoperability  is  a  luxury  rather  than  a  require¬ 
ment  for  sound  management. 

Lack  of  Standards  Across  the  Board.  There  are  still 
major  technological  areas  that  have  not  as  yet 
promulgated  data/information  standards. 

Output  Definition.  As  a  follow  on  to  the  Generator 
vs.  Users  bullet  above,  there  are  requirements  for 
the  output  of  data/information  in  highly  different 


ways  to  meet  the  specific  needs  of  a  given  group  of 
stakeholders.  This  involves  multiple  issues  of 
information  management  and  interoperability  so 
users  can  obtain  the  output  type(s)  of  choice. 

The  bottom  line  is  that  there  is  a  lack  of 
interoperability  as  one  goes  back  and  forth  through  the 
data-information-knowledge  spectrum  shown  in  Figure  2. 


The  desired  future  is  one  of  fully,  across-the-board 
interoperable  information  management  services 
available  to  the  broadest  range  of  stakeholders  possible. 
The  desired  information  environment  is  shown  in 
Figure  3.  The  desired  future  involves  seamless  move¬ 
ment  of  data,  information,  and  management  from 
research  to  analysis  to  synthesis  to  adaptation.  This  is 


Vision  for  the  future 


Figure  3.  Desired  future  state  of  information  management  interoperability. 
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particularly  true  in  the  decision  science  area  where  new 
science  findings  must  flow  to  implementation,  and 
where  feedback  produces  the  need  for  adaptation  of  a 
given  management  decision. 

Toward  this  end,  there  are  five  areas  that  are  viewed 
as  desired  levels  of  functionality  for  future 
interoperable  information  systems.  These  are: 

1.  Access:  Differing  stakeholders  utilize  different 
components  of  an  interoperable  information 
system  from  their  geograhically-distributed 
desktops. 

2.  Catalog:  Provide  for  a  repository  for  techniques/ 
algorithms  in  a  standard,  web-searchable 
paradigm. 

3.  Smart  agent/knowledge  management:  Methods  to 
capture  the  knowledge  base,  sharing  “lessons 
learned,”  “decision  model”  “case  studies,”  etc. 
for  repositing  in  the  cataloging  framework  above. 

4.  Motivation:  Develop  mechanisms  that  strongly 
encourage  the  development  of  in-house  and 
marketplace  interoperable  solutions. 

5.  Processing:  Interoperability  and  reuse  of  tools 
through  standards. 

The  desired  future  would  have  the  following  goals/ 
objectives/features. 

•  All  data  would  have  standardized  repository  by 
data  type; 

•  Users  would  obtain  and  manage  data  via  integrated 
problem  solving  environments; 

•  Interoperability  and  reuse  of  tools  promoted 
through  standard  use  of  standards; 

•  Smart  agents  facilitate  input  requirement,  tool 
selection  &  results  analysis; 

•  Computational  environment  facilitates  sharing 
“lessons  learned,”  “algorithm,”  “decision  model,” 
“case  studies”; 

•  Repository/tracking  of  decision  process  facilitated 
through  problem  solving  environment; 

•  One  set  of  national  interoperability  standards 
utilized; 

•  Financial  incentives  for  contractors/vendors  to  use 
standards  set  in  place; 

•  Eliminate  designed  non-interoperability  (“not 
invented  here”); 

•  Provide  a  higher  conceptual  or  natural  language 
query  capability  to  facilitate  decision  support  to 
many  stakeholders;  and, 

•  Make  new  technology  implementations  transparent 
to  the  user. 

This  set  of  goals/objectives/features  would  apply 
across  the  spectrum  listed  in  Figure  3.  This  alone  would 
promote  interoperability  in  a  meaningful  way. 


Strategies  for  implementation 


The  desired  future  is  achievable  through  strategic 
implementation  activities  as  listed  below.  It  is  noted  that 
this  list  is  incomplete  both  in  scope  and  in  specifics. 
Ftowever,  it  is  clear  that  the  proposed  implementation 
strategy  includes  marketplace,  organizational,  technical, 
and  programmatic  factors,  any  of  which  can  frustrate 
convergence  to  the  desired  future  state.  The  implemen¬ 
tation  concepts  are  keyed  to  the  desired  future  state 
bullets  listed  above. 

•  All  data  would  have  standardized  repository  by 
data  type — develop  open  “wizards”  that  facilitate 
placing  data  within  web-accessible  repositories. 
Define  these  repositories  using  standards  as  well  so 
that  they  can  be  queried  and  mined  by  different 
users  who  see  them  as  virtual  repositories. 

•  Users  would  obtain  and  manage  data  via  integrated 
problem  solving  environment — a  consistent, 
integrated  problem  solving  computational  environ¬ 
ment,  a  “decisioning”  world  on  the  web,  is  needed. 
Major  Federal  government  organizations  and 
industry  should  collaborate  to  provide  the  critical 
mass  needed  to  properly  develop  the  environment. 
Note  that  this  is  not  to  suggest  the  development  of 
a  single  environment  with  one  look  and  feel,  but  a 
global  environment  with  a  user-customizable 
toolkit  that  builds  off  a  common  set  of  information 
technologies  and  standards. 

•  Interoperability  and  reuse  of  tools  promoted 
through  standard  use  of  standards — this  goal  can 
be  realized  through  the  use  standards,  development 
of  repositories,  development  of  the  problem 
solving  environment  listed  above,  and  through 
several  of  the  items  listed  below. 

•  Smart  agents  facilitate  input  requirement,  tool 
selection  and  results  analysis — develop  a  standard 
set  of  agents,  acting  behind  the  scenes  over  a 
network  or  the  web,  to  facilitate  data/information 
creation,  location,  and  retrieval,  model/tool 
selection,  visualization,  etc.  These  agents  would  be 
“freeware”  to  insure  their  broad-based  use. 

•  Computational  environment  facilitates  sharing 
“lessons  learned,”  “algorithm,”  “decision  model,” 
“case  studies” — this  would  be  an  extension  of  the 
user  environment  above  to  some  extent,  but  would 
also  employ  smart  agents  and  the  creation  of 
publication  standards  to  mine  and  reposit  new 
findings  and  information  into  repositories  in  object 
formats. 

•  Repository/tracking  of  decision  process  facilitated 
through  problem  solving  environment — a  new  set 
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of  smart  agents  or  wizards  should  be  established  to 
monitor  and  feedback  the  actual  processes  used  in 
decision  making  for  a  given  situation. 

•  One  set  of  national  interoperability  standards 
utilized — this  should  be  an  established,  focused, 
directed,  and  funded  objective  of  the  major  federal 
agencies,  with  incentives  to  industry,  to  insure  that 
one  national  set  of  standards  is  created.  The 
referenced  incentives  are  needed  to  insure  that 
these  standards  become  the  marketplace  standards 
as  well  to  insure  life-cycle  use  and  implementa¬ 
tion. 

•  Financial  incentives  for  contractors/vendors  to  use 
standards — major  procurement  organizations  (e.g., 
government)  should  provide  financial  incentives 
through  contracts  to  promote  (require?)  the  use  of 
established  standards.  Other  marketplace  incen¬ 
tives,  such  as  the  use  of  cooperative  agreements 
combining  government  and  private  funding  to 
develop  standards,  should  be  employed. 

•  Eliminate  designed  non-interoperability  (“not 
invented  here”) — promote  the  use  of  standards 
through  Agency  order  at  the  national  or  agency 
level. 

•  Provide  a  higher  conceptual  or  natural  language 
query  capability  to  facilitate  decision  support  to 
many  stakeholders — develop  the  means  for 
decision  makers  to  query  data  of  all  types  (includ¬ 
ing  modeling  and  simulation  data)  using  terms 
(language)  they  understand  and  identify  with. 

•  Make  new  technology  implementations  transparent 
to  the  user — this  would  be  done  primarily  through 
the  use  of  object-oriented,  modular  developments 
and  the  use  of  standards/agents  as  listed  above. 

In  addition  to  the  points  raised  above,  it  is  essential 
that  a  paradigm  be  established  that  encourages  the 
differing  stakeholders  within  the  decision  process  to 
conduct  the  activities  at  which  they  excel  rather  than  all 
the  components  of  the  process  per  se.  For  example,  the 
development  of  an  interoperable  information  technol¬ 
ogy  architecture  envisioned  might  cause  one  to  con¬ 
clude  that  decision  makers  with  limited  technical 


background  could  simply  go  to  the  web,  access  data, 
execute  models,  and  perform  visualization  at  will.  This 
should  not  be  either  the  expectation  or  the  goal.  Rather, 
the  interoperable  model  presented  above  should  be  used 
to  facilitate  the  interactions  between  experimentalists, 
modelers,  integrators,  analysts,  managers,  and  the 
public  rather  than  replace  said  interactions.  Toward  this 
end,  it  is  essential  that  the  process  of  creating  the 
envisioned  interoperable  environment  also  include 
checks  and  balances  that  facilitates  the  science  and 
engineering  community’s  verification  of  tools/models/ 
methods  prior  to  their  use  by  decision  makers — and  that 
stakeholders  have  the  opportunity  to  directly  frame  the 
use  of  said  tools  in  deciding  alternative  futures  for  a 
given  site. 


Recommendations 


It  is  tempting  to  recommend  simply  that  the  items  listed 
above  that  are  required  to  achieve  the  desired  future  be 
funded  and  conducted  as  presented.  However,  there  is 
significant  investigation  and  discovery  left  to  conduct  to 
understand  the  best  ways  to  implement  the  goals/ 
objectives  listed  above. 

Toward  that  end,  it  is  recommended  that  a  series  of 
highly-focused  technology  demonstrations  be  created 
and  conducted  that,  by  design,  investigate  the  better  of 
several  means  for  achieving  interoperable  information 
technologies.  Envisioned  is  a  set  of  five  to  seven 
demonstrations,  chosen  to  reflect  differing  natural  and 
water  resources  decision  processes,  each  with  differing 
stakeholder  requirements,  that  exercises  the  differing 
interoperability  and  information  aspects  listed  above.  It 
is  recommended  that  the  Federal  government  agencies 
with  major  roles  in  the  natural  and  water  resources  area 
come  together  to  establish  these  demonstrations  in 
direct  concert  with  stakeholder  groups  and  industry. 

The  time  period  for  the  demonstrations  should  be  no 
more  than  two  years. 
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Knowledge  Management  and  Decision  Support 

6/ Wayne  Schmidt,  U.S.  Army  Engineer  Research  and  Development  Center,  Construction  Engineering  Research 
Laboratory,  Champaign,  IL  61826-9005 


Introduction 


“If  I  have  only  a  hammer,  the  world  looks  like  a  nail  ” 
As  the  number  and  complexity  of  environmental 
analysis  tools  grows  and  the  decision  process  become 
more  complex,  it  becomes  increasing  difficult  to  keep 
current  on  the  best  methods.  This  paper  envisions  how 
the  emerging  field  of  knowledge  management  can  be 
used  to  assist  in  the  applications  of  analytical  tools,  data 
representation  and  modeling  to  help  the  user  solve 
problems  related  to  natural  resources  and  the  environ¬ 
ment.  This  paper  describes  the  current  state  of  the  art 
for  knowledge  management,  a  vision  of  how  knowledge 
can  be  managed  to  support  natural  resource  and 
environmental  decision  making,  and  a  strategy  for 
demonstrating  the  vision. 

Why  is  knowledge  management  important? 

The  term  Knowledge  Management  (KM)  has  become  a 
key  issue  for  government,  industry  and  certainly 
Information  Technology  (IT)  executives.  Organizations 
are  becoming  increasingly  aware  of  the  importance  of 
managing  knowledge,  like  any  other  asset,  to  improve 
their  competitive  advantage.  Careful  application  of 
knowledge,  like  other  assets,  can  result  in  better 
decisions,  particularly,  at  the  working  level.  Typically, 
there  is  a  wide  variety  of  analytical  tools  available.  The 
users  dilemma  often  becomes  one  of  managing  the 
entire  decision  support  process.  Restated,  “What  is  the 
sequence  of  tasks  that  will  reliably  produce  the  report/ 
documentation  to  support  the  conclusions  reached.” 


What  is  knowledge? 

Before  one  can  talk  about  knowledge  management,  it  is 
useful  to  have  an  understanding  of  how  knowledge 
differs  from  information  or  data.  Peter  Drucker  defines 
knowledge  as  “Information  that  changes  something  or 
somebody — either  by  becoming  grounds  for  actions  or 


by  making  an  individual  (or  an  institution)  capable  of 
different  or  more  effective  action.” 

Knowledge  is  the  result  of  aggregating  process, 
information,  analysis,  and  supporting  documentation 
into  a  package  that  will  support  a  conclusion.  It  is  the 
end  result  of  combining  these  factors  and  our  experi¬ 
ence.  The  concept  of  transforming  data  into  information 
is  well  known  and  understood.  This  concept  can  be 
extended  to  characterize  knowledge  as  part  of  a 
relationship  pyramid  (Figure  4).  The  pyramid  illustrates 
how: 

•  Data  in  context  yields  information; 

•  Information  after  analytical  effort  yields  under¬ 
standing; 

•  Understanding  when  combined  with  professional 
judgment  yields  knowledge;  and, 

•  Knowledge  in  turn  supports  decision-making. 

There  are  a  wide  variety  of  analytical  tools  available. 

The  tools  assist  the  user  in  moving  from  information  to 
understanding.  Information/data  is  processed  and 
displayed  in  a  variety  of  forms  and  analyzed  by  various 
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models  to  present  data  in  a  way  that  can  be  easily 
understood.  While  this  is  an  important  part  of  the  entire 
decision  making  process,  there  are  many  other  tasks 
that  must  be  accomplished.  For  example,  there  is  data  to 
be  gathered,  presentations  to  be  made,  reports  to  be 
made,  decision  papers,  executive  summaries,  public 
announcements  etc.  All  of  these  tasks  are  part  of  the 
knowledge  that  must  be  managed  to  aid  the  user  in 
completing  the  decision  process. 


What  is  knowledge  management? 

Knowledge  Management  is  an  integrated,  systematic 
approach  to  identifying,  managing  and  sharing  an 
enterprise’s  information  assets,  including  documents, 
databases,  policies,  procedures,  and  implicit  expertise. 
The  purpose  of  this  integration  is  to  make  available  the 
validated  “Decision  Support  Process”  that  will  guide  the 
user  to  the  decision  point.  In  many  organizations  this 
Decision  Support  Process  is  well  understood  for  a 
particular  class  of  problems.  However,  when  new 
problems  arise,  or  institutional  knowledge  is  lost  due  to 
retirement,  transfer,  etc.,  effectiveness  suffers  while  the 
process  is  relearned. 

The  level  of  technical  ability  of  stakeholders  within 
the  decision  process  is  often  very  different.  The  citizen 
will  have  the  least  technical  ability  to  understand  and 
manipulate  the  data.  The  decision  maker  has  a  better 
understanding,  the  engineering  staff  has  still  more  and 
finally  the  research  modelers  have  the  best  understand¬ 
ing  of  the  data.  However,  the  researcher  is  removed 
from  the  decision  process  and  is  often  solely  concerned 
with  analysis.  The  users  in  the  shaded  area  are  the 
intended  users  of  a  knowledge  management  capability. 
They  understand  the  nature  of  the  decision,  but  don’t 
have  the  knowledge  to  plan  and  manage  the  Decision 
Support  Process.  In  short,  they  need  a  knowledge 
management  capability  (Figure  5). 

State-of-the-art  in  knowledge 
management 

Knowledge  management,  while  the  subject  of  many 
conferences,  books,  papers  and  tools,  is  immature. 

There  are  tools  that  accomplish  some  knowledge 
management  but  do  not  address  the  entire  concept. 

Traditionally,  libraries  provide  the  knowledge 
management  function.  Libraries  store  the  accumulated 
knowledge  of  the  ages.  The  time-honored  way,  espe¬ 
cially  in  the  scientific  community,  is  to  make  knowl¬ 
edge  explicit  by  publishing.  However,  most  of  the 


scientific  literature  is  very  technical  and  only  useful  to 
other  scientists,  not  the  problem  solvers.  It  also  deals 
with  theory  and  not  the  tasks  that  are  required  to 
complete  the  decision  process.  In  addition,  it  is  paper 
based  and  difficult  to  reuse. 

The  publishing  process  is  long  and  resource  inten¬ 
sive.  It  is  seldom  used  in  the  business  world  and  notably 
not  used  to  make  the  business  process  explicit.  Some 
businesses  publish  “Standard  Operative  Procedures” 
and  “Policies.”  Keeping  these  printed  documents 
current  is  a  resource  intensive  problem.  Thus,  they 
become  obsolete  and  fail  to  be  effective  sources  of 
knowledge.  Again,  they  are  usually  paper-based. 

Many  tools  are  available  to  deal  with  parts  of  the 
knowledge  management  problem.  These  tools  are  often 
called  knowledge  management  solutions  but  only 
address  part  of  the  problem.  For  example,  there  are  very 
complex  search  engines  that  will  search  the  web,  your 
stored  documents,  email  and  even  databases.  But  they 
don’t  address  why  you  want  this  information,  where  it 
fits  within  the  decision  support  process  and  where  an 
approved  method  of  reaching  a  conclusion  exists. 

The  technologies  for  Knowledge  Management 
revolve  around  implementing  the  KM  Process.  Each  of 
the  stages  has  existing  technologies,  although  many  are 
marketed  under  the  global  “knowledge  management” 
title.  Some  technologies  span  several  of  the  stages;  but 
addressing  the  entire  spectrum  requires  careful  thought 
and  planning.  The  existing  technologies  include: 

•  Process  Development: 

♦  IDEF  models  (approved  Federal  Government 
Std) 

♦  Rummler-Brache 
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•  Search: 

♦  Search  engines  (Infoseek,  web  crawler) 

♦  Information  portals  (Yahoo,  Altavista,  DogPile) 

♦  Databases  (e.g.,  Lexis-Nexis) 

♦  Information  Systems  (GIS) 

♦  Data  mining  and  warehousing 

♦  Expertise  tracking  and  locating 

•  Organize: 

♦  Decision  Support  Systems 

♦  Word  processors 

♦  Knowledge  Mapping 

♦  Workflow 

•  Create: 

♦  Group  Decision  Support  Systems 

♦  Collaboration  Portals 

♦  Discussion  groups 

♦  Video  Teleconferencing 

♦  Storytelling 

•  Capture: 

♦  Peer  review 

♦  Cataloging  and  indexing 

♦  Business  practice  repository 

♦  Documents  designed  for  retrieval  by  coding 
content  (XML) 


A  DSP  is  a  linked  set  of  tasks  that  describe  how  to 
achieve  a  solution  (Figure  6).  Some  of  the  tasks  will 
require  manual  action,  e.g.,  “coordinate  with  the  EPA.” 
Others  will  require  the  use  of  word  processors,  e.g., 
“Develop  a  status  statement  for  the  public  affairs 
office.”  Still  others  will  suggest  specific  tools  to 
accomplish  data  analysis. 

Each  DSP  has  these  attributes: 

•  Decision/problem  addressed 

•  Description 

•  Where  used  and  when 

•  Developer  of  the  DSP 

•  Relevant  policies 

•  Tools  used 

•  Start  conditions 

•  Data  needed  (minimum) 

•  Products  produced 

•  References 

The  DSP  library  is  searchable  by  these  attributes. 
Thus,  the  user  can  search  for  all  uses  of  a  particular 
tool.  For  an  expert  user,  the  selection  of  analytical  tools 
may  be  critical.  Selection  of  a  tool  will  then  drive  a 
reformulation  of  the  DSP. 


Vision  for  the  future 


Strategy  for  implementation 


Any  vision  for  knowledge  management  must  be 
focused  on  a  user  community.  Our  knowledge  manage¬ 
ment  vision  is  from  the  perspective  of  the  user  commu¬ 
nity  described  earlier.  Thus,  the  vision  is:  A  searchable 
distributed  library  of  Decision  Support  Processes 
(DSP).  Each  DSP  is  a  complete  “case  study”  of  the 
process,  data  requirements,  analytical  tools,  and  report 
examples  necessary  to  propose  and  support  decision 
alternatives. 


•  Define  a  model  for  a  DSP.  This  would  include 
attributes  and  methods  for  combining  DSPs  and 
updating  them  as  tools  change. 

•  Develop  a  case  study.  Build  a  set  of  DSPs  for  a 
particular  problem  domain. 

•  Develop  a  search  and  presentation  scheme  for 
users. 

•  Integrate  a  lessons  learned  capability  and  a  way 
for  users  to  evaluate  the  DSPs. 


Figure6.  Decision  support  process. 
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Develop  a  web  site  to  improve  user  access. 
Address  the  issue  of  getting  new  DSPs.  Perhaps 
there  could  be  contractual  language  that  would 
require  contractors  to  input  their  process  and 
results  in  an  electronic  form  in  addition  to  the 
normal  printed  documents. 


In  summary 


Fundamentally,  knowledge  management  makes  the 
collective  information  and  experience  of  an  enterprise 
available  to  the  individual  knowledge  worker,  who  is 
responsible  for  using  it  wisely  and  for  replenishing  the 
knowledge  asset.  This  ongoing  cycle  promotes  a 
learning  organization,  stimulates  collaboration  and 
empowers  people  to  continually  enhance  the  way  they 
perform  work. 
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Computation,  Communication  and  Data  Storage 

By  Doug  Johnston,  Senior  Research  Scientist,  National  Center  for  Supercomputing  Applications,  University  of 
Illinois  at  Urbana-Champaign 


Introduction 


What  is  issue/objective? 

The  rapid  development  of  higher  and  higher  perfor- 
mance  computing  and  communications  seems  to 
continue  unabated.  Power  that  exceeds  supercomputers 
of  ten  to  fifteen  years  ago,  costing  millions  of  dollars 
are  now  available  on  the  desktop  for  less  than  $2,000. 
Network  communications  used  by  a  small  number  of 
academic  and  government  researchers  is  now  accessible 
to  virtually  all  people  through  browser  and  Internet 
technologies.  While  there  is  little  doubt  that  increasing 
scalable  networking  and  computation  power  is  being 
developed,  there  is  also  a  concern  that  there  is  a 
deficiency  in  comparable  development  in  language, 
tools,  and  interface  environment  to  make  this  power 
usable  to  a  broader  (i.e.  non-scientist)  community. 
Usable  power  refers  to  making  access  to  the  computa¬ 
tional  resources  easier,  by  reducing  or  removing  the 
considerable  barriers  that  remain. 

The  barriers  to  more  effective  use  in  the  decision 
making  environment  remain  numerous.  The  information 
environment  remains  disorganized  with  relevant  data 
distributed  across  many  databases  and  systems  with 
differing  descriptions  based  on  disciplinary  focus. 
Models  for  analysis  or  creation  of  data  are  similarly 
distributed  and  disciplinary.  Thus  decision  makers, 
confronted  with  a  problem  to  solve,  must  seek  out 
relevant  data,  acquire  that  data,  transform  it  to  a 
common  framework,  identify  the  models  that  will 
address  the  problem  being  solved,  acquire  the  computa¬ 
tional  resources  required  to  run  the  models,  and  finally, 
if  all  goes  well,  actually  begin  to  address  the  real 
problem  they  set  out  to  solve. 

The  objective  of  technology  applied  to  decision 
making  then,  is  to  make  available  resources  (computa¬ 
tional,  data,  model)  at  the  right  time  applied  to  the  right 


problem.  The  focus  of  this  discussion  is  on  methods  for 
making  resources  available  to  the  user. 

Status 

In  many  respects,  the  technological  advances  in  the  last 
few  decades,  and  in  the  last  few  years,  have  greatly 
enabled  access  to  information  resources.  The  rapid 
growth  of  distribution  of  information  over  the  Internet 
and  access  through  Web  browsers  is  certainly  an 
indication  of  that.  At  the  same  time  however,  it  has 
enforced  the  realization  that  physical  access  to  informa¬ 
tion  through  networking  is  not  the  same  as  usable 
access.  The  proliferation  of  information  on  the  World 
Wide  Web  has  shown  how  unorganized  and  undocu¬ 
mented  the  information  world  truly  is.  Focusing  only  on 
geographic  information  resources,  finding  data  for  the 
location  of  interest,  at  the  necessary  scale,  using  an 
appropriate  attribute  domain,  is  itself  a  major  challenge. 
The  growth  of  clearinghouses  and  data  catalogues  is  a 
step  toward  overcoming  some  of  these  barriers,  and 
research  efforts  in  digital  library  technologies  promise 
advances  as  well. 

In  decision  making,  data  are  only  part  of  the  equa¬ 
tion.  To  assess  the  impact  of  events  or  predict  the 
outcomes  of  alternative  plans  and  policies,  models  are 
required  to  represent  the  processes  or  phenomena  of 
interest.  Models,  built  by  scientists,  can  be  extremely 
rich  but  can  also  be  extremely  opaque  to  a  potential 
user  who  has  the  expertise  to  use  the  information 
generated  by  the  models,  but  possibly  not  the  disciplin¬ 
ary  focus  of  the  modeler.  Arguably,  many  science 
models  are  built  by  scientists  for  scientists. 

The  world  is  also  seeing  a  tremendous  growth  in  the 
amount  of  data  available.  This  is  partly  because  more 
data  are  being  stored  in  accessible  (i.e.,  digital)  forms, 
but  also  because  more  data  are  being  collected.  The 
launch  of  the  new  generation  of  remote  sensing  plat¬ 
forms  promise  wide  coverage  at  very  high  resolution, 
but  the  price  of  that  is  the  tremendous  growth  in  the 
volume  of  data  that  needs  to  be  stored  and  analyzed. 
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While  the  raw  volume  of  storage  capacities  of  magnetic 
drives  and  off-line  storage  continues,  the  issue  of 
searching  and  rapidly  accessing  that  information 
remains. 

The  development  of  common  interface  protocols 
developed  through  the  PC  “revolution”  and  more 
recently  web  browser  technology  for  the  Internet 
“revolution  ”  has  increased  the  provision  of  uniform 
access  to  resources  by  a  larger  audience  (whose 
principle  business  is  not  the  computational  technology). 
Communications  protocols  such  as  TCP/IP  have 
enabled  higher  levels  of  development  built  on  top  of 
these  foundational  technologies. 

There  is  no  question  that  there  are  good  computers, 
good  models  and  good  data  available,  but  in  many  cases 
they  are  developed  for  single  purposes  by  and  for  single 
disciplines.  The  fundamental  issue  in  many  cases  is  not 
a  technical  one,  but  an  institutional  one  of  choosing  to 
adopt  standards,  use  protocols,  provide  access,  etc. 
Nonetheless,  technological  advances  in  the  infrastruc¬ 
ture  development  can  facilitate  use  of  these  formidable 
resources.  The  question  is,  how  do  we  make  models/ 
data  accessible  to  a  broader  user  community? 


Needs/requirements 

The  problems  of  access  to  computational  and  informa¬ 
tion  resources  suggest  that  integration  or  logical 
connection  between  resources  is  a  desired  focus  area. 
The  following  needs/requirements  are  proposed  as  areas 
of  high  priority. 

DATA 

•  Cohesive  data  sets  linked  independently  of 
individual  projects. 

•  Data  extraction  and  discovery  tools 

•  Uniform  means  to  access  large  data  sets 
DATA/MODELS/INTERFACE 

•  Integrated  Search  capabilities 

•  Integration  of  storage  media 

♦  Primary  (on  line) 

♦  Secondary  (disk  backup) 

♦  Tertiary  (tape  storage) 

MODELS 

•  Model  definition  language 

•  Data  input  output  specification 

•  Model  documentation  for  suitability  for  use 
INTERFACE 

•  Uniform/consistent  understanding  of  available 
resources  and  access 

•  Push  technologies  (to  notify  of  changes,  etc). 

•  Paradigms  and  tools  to  deal  with  large,  hierarchical 
datasets,  studies,  knowledge. 


State  of  the  art  computation, 
communication  and  data  storage 


Large  storage 

There  exist  many  approaches  to  management  and 
storage  of  large  datasets,  including,  on  the  hardware 
side,  Giga-  and  Terabyte  primary,  secondary,  and 
tertiary  storage  systems,  and  on  the  software  side, 
several  high-performance  commercial  systems  such  as 
Oracle,  Informix,  MSQL,  etc.  The  principle  issue  in  this 
arena  is  defining  a  mechanism  for  robust  and  efficient 
access  to  these  systems.  Network  bandwidth  available 
to  the  user,  physical  access  to  storage  devices  by  users, 
overhead  costs  with  maintaining  large,  complex 
datasets,  data  input  output  bottlenecks  from  storage  to 
processor  are  a  few  of  the  issues  that  need  to  be 
addressed  to  keep  up  with  the  advances  in  online  and 
archival  storage. 

These  issues  of  course,  are  not  unique  to  Federal 
Agency  Decision  Support  so  there  are  many  efforts 
throughout  the  CS  community  to  address  these  includ¬ 
ing  efforts  such  as  the  Hierarchical  Data  Format  (HDF) 
(www.hdf.ncsa.uiuc.edu)  developed  for  NASA, 


Simulation  models  for  science 

Many,  many  models  exist  for  various  purposes.  Even 
models  for  the  same  purpose  employ  different  theoreti¬ 
cal  concepts  of  physical  or  social  processes,  different 
numerical  algorithms  for  solving  applications  of  the 
theoretical  concepts,  and  certainly  different  program¬ 
ming  languages,  file  types,  data  sources,  etc.  to  execute 
the  models.  There  is  no  question  that  there  are  good 
models  and  tools  for  a  wide  range  of  application,  and 
better  ones  (higher  resolution,  capturing  finer  and  finer 
details  of  the  phenomena,  greater  efficiency,  etc)  but  the 
issue  remains  from  a  users  or  market  perspective  of 
being  to  choose  among  them,  or  being  able  to  assemble 
components  in  order  to  construct  a  system  for  address¬ 
ing  a  particular  class  of  problem  that  may  span  pro¬ 
cesses.  At  a  foundational  level,  there  are  many  tech¬ 
nologies  being  employed  to  facilitate  the  construction 
of  complex,  scientific  models  including  the  Modular 
Modeling  System,  DIAS,  etc.,  to  name  a  very  few. 


Web  Browsers 

It  is  easy  to  forget  the  birth  of  widespread  use  of  the 
internet,  but  the  general  public  and  all  the  e-commerce 
and  other  industries  spawned  from  it,  started  in  the 
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1 980’s  with  the  world  wide  web  consortium  and  in 
1993  with  the  release  of  Mosaic,  the  first  multifunction 
web  browser.  The  development  of  web  browsers,  e- 
mail,  on-line  transactions,  exchange  of  multi-media 
documents  (and  the  tools  to  use  them)  and  other 
technologies  has  enabled  tremendous  growth  in  the 
range  of  services  available  through  the  net  (and  the 
expectations  associated  with  it).  At  the  same  time,  the 
proliferation  of  search  engines,  agents  and  other  tools  to 
find  information  show  how  difficult  it  currently  is  to 
find  just  the  right  piece  of  information.  While  highly 
structured  information  classification  systems  may  limit 
the  richness  of  information  query,  unstructured  classifi¬ 
cation  systems  cannot  guarantee  access  to  information, 
even  if  it  exists  within  the  system. 

The  bottleneck  to  improved  use  of  information  and 
computation  is  argued  here  to  be  less  an  issue  of 
needing  greater  advances  in  technological  development 
(although  technological  development  is  critical)  but 
more  an  issue  of  finding  ways  to  increase  the  rate  of 
adoption  of  existing  and  emerging  technology  by 
decision  makers.  The  current  state  is  illustrated  in 
Figure  7.  Components  of  the  decision  making  environ¬ 
ment  are  shown  as  corners  of  the  pyramid.  They  are 
identified  as  the  data  base,  the  model  base,  the  compu¬ 
tation  base,  and  the  user  base.  The  volume  of  the 
pyramid  represents  the  structure  that  links  these 
components  together.  The  solid  gray  corners  are 
representative  of  the  current  state  of  development.  In 
other  words,  we  argue  that  the  individual  sectors  (data 


User  Base 


Model  Base 

Figure  7.  State  of  decision-making  technology  environment. 


base  technologies,  modeling  tools,  computational 
resources,  and  user  interfaces)  have  each  seen  develop¬ 
ment,  but  the  filling  the  pyramid,  and  the  full  integra¬ 
tion  of  the  resources  remains  incomplete. 


Vision  for  the  future 


There  are  two  (NOT  mutually  exclusive)  strategies  for 
completing  the  structure.  One  strategy  is  to  continue 
building  from  the  corners,  by  developing  faster  comput¬ 
ers,  larger  databases,  better  models,  and  better  user 
interfaces.  Another  strategy  is  to  create  a  new  node  in 
the  center  and  build  outward  to  join  the  foundational 
technologies  where  they  currently  exist,  and  co-develop 
along  with  these  technologies.  This  node  is  termed  a 
broker  and  represents  a  tiered  set  of  services  built  to 
connect  the  technologies  in  ways  that  can  bring  existing 
capabilities  to  bear  fruit  quickly. 


Broker 

The  broker  functionality  is  borrowed  from  the  familiar 
services  sector.  A  broker  provides  a  means  of  filtering 
through  available  options  and  presenting  a  more  limited 
set  of  comparable  options  for  subsequent  action.  The 
architecture  for  this  broker  service  (Figure  8)  consists 
of  a  set  of  components  providing  the  information 
needed  to  filter  through  the  information.  It  should  be 
noted  that  some  of  the  barriers  presented  above  exist 
here,  but  the  emphasis  here  is  on  confronting  those 
barriers  directly  by  providing  a  tiered  approach  to 
implementation. 


Catalogs 

The  foundational  element  of  the  broker  function  is  the 
construction  of  catalogues  providing  information  on  the 
available  elements  (products).  The  catalog  contains  the 
attributes  of  the  elements  and  a  method  for  querying 
these  attributes. 


Abstraction 

The  definition  of  attributes  of  catalog  elements  requires 
the  creation  of  model,  data,  and  computation  abstrac¬ 
tions,  that  is,  generalizable  descriptions  of  relevant 
features  that  form  the  structure  of  the  query  and  can  be 
populated  by  the  producers  of  the  elements.  The 
abstraction  is  independent  of  specific  implementations 
(programming  languages,  units  of  analysis,  dates  of 
record,  etc.). 
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Figure  8.  Broker  services  architecture. 


Definition  Translator 

It  is  assumed  that  it  is  not  possible  (or  even  desirable)  to 
define  or  enforce  a  universal  standard  on  model/data/ 
computation  specification  or  implementation.  Thus  to 
map  the  abstract  definition  to  the  actual  elements,  a 
definition  translator  is  required.  In  a  shopper/catalogue/ 
warehouse  analogy,  the  definition  translator  may  take 
the  product  identification  generated  from  the  catalogue 
description  and  returns,  for  example,  what  resources 
might  be  required  to  ship  it,  such  as  size,  weight,  cost, 
special  handling,  etc.). 


Store 

The  data  or  model  store  is  characterized  not  as  a 
monolithic  data  depository  but  rather  a  set  of  pointers  to 
the  actual  resources.  In  a  warehouse  analogy,  the  store 
takes  a  particular  product  identifier  and  tells  the 
warehouse  operator  where  that  product  is  located. 


Strategies  for  implementation 


A  multi-tiered  strategy  for  advancing  progress  in  this 
area  is  suggested.  An  overriding  objective  is  to  ensure 


that  the  decision  support  community  is  engaged  in  the 
process  of  defining  and  developing  this  functionality. 
The  strategy  is  a  market  or  user-focused  approach  to 
bridging  the  gaps  between  existing  technologies. 


Tier  1 

•  Literature  review  on  model  definition  language 
Data  definition  languages  through  metadata 
specifications  and  interoperability  activities  are 
advancing  rapidly.  A  similar  effort  for  models 
needs  to  be  launched. 

•  Test  against  marketplace 

Arguably,  one  of  the  barriers  to  decision  support 
technology  is  the  perceived  lack  of  market  poten¬ 
tial  by  private  sector  technology  providers. 

•  Government  influence  on  technology  trajectory 
The  creation  of  broker  services  can  be  disabled  by 
unwillingness  to  participate,  even  if  the  functions 
perform  to  specification.  As  these  efforts  develop, 
the  government  can  assist  by  including  require¬ 
ments  in  contracts  and  bids  supporting  participa¬ 
tion. 

•  Enable  access  to  data 

In  support  of  capturing  “low  hanging  fruit,” 
demonstration  functions  supporting  brokered 
access  to  distributed  data  should  be  supported. 
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This  can  occur  through  existing  demonstration 
projects  undertaken  by  the  federal  agencies  and 
need  not  necessarily  require  new  programs. 


Tier  2 

•  De-couple  models 

•  Formalize  Definition  Models 

•  Basic  Broker  Functionality  (search/discovery) 

•  Maintain  connection  with  CS  community  re  GRID 
development 

Tier  3 

•  Advanced  Broker  Function 

•  Implement  within  emerging  computational  GRID 
environment  (second  generation  Web) 


Recommendations 


1.  Workshop  aimed  at  defining  model  components 

•  Model  Description  Language 

•  Based  on  fundamental  cleavages  between  types 

•  Look  at  ERDC  Model  catalog 

2.  Form  multilateral  evaluation  group  (e.g.,  OGC) 

3.  Make  case  for  market 

4.  Need  a  synoptic  program/for  teaching  and 
research 

5.  Consortia/partnership  of  compute  service 
providers/ISP/and  ASP 


Roles 


Government  specs  and  requirements 
Consortia  -  standards 
Industry  -  widgets 
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